Because of mathjax bug

The Duckietown Book (Duckiebook)

✎🔗

From kits of parts to an autonomous fleet
in 857 easy steps
without hiding anything

The last version of this "duckiebook" and other related documents are available at the URL
http://book.duckietown.org/

For searching, see the one-page version.

Table of contents

✎🔗
Because of mathjax bug
Because of mathjax bug

The Duckietown project

✎🔗

What is Duckietown?

✎🔗

Duckietown history and future

✎🔗

Duckietown classes

✎🔗

Duckietown is an international effort. Many educational institutions have adopted the platform and used to teach robotics and related subjects. If you have used Duckietown in any of your course and cannot find a mention to it here, contact us.

add contact email

Here, we provide a chronologically ordered compendium of the Duckietown related learning experiences worldwide.

2016

✎🔗

Duckietown was created in 2016.

Massachusetts Institute of Technology

✎🔗

Location: United States of America, Cambridge, Massachusetts

Course title: 2.166 Vehicle Autonomy

Instructors: plenty

Educational Level: Graduate

Time period:

Link:

Highlights: Role-playing experience (Duckietown Engineering Co.), public demo

Summary:

Duckietown at MIT in 2016

National Chiao Tung University

✎🔗

Location: Hsinchu, Taiwan

Course title: Robotic Vision

Instructors: Prof. Nick Wang

Educational Level: Graduate

Time period:

Link:

Summary:

Highlights:

Tsinghua University

✎🔗

Location:

Course title:

Instructors:

Educational Level:

Time period:

Link:

Summary:

Highlights:

Rennselaer Polytechnic Institute

✎🔗

Location:

Course title:

Instructors:

Educational Level:

Time period:

Link:

Summary:

Highlights:

First steps

✎🔗

Duckietown for instructors

✎🔗

to write

Duckietown for self-guided learners

✎🔗

to write

Introduction for companies

✎🔗

to write

Frequently Asked Questions

✎🔗

Duckumentation documentation

✎🔗
Because of mathjax bug

Contributing to the documentation

✎🔗

Installing the documentation system

✎🔗

In the following, we are going to assume that the documentation system is installed in ~/duckuments. However, it can be installed anywhere.

We are also going to assume that you have setup a Github account with working public keys.

Basic SSH config.

Key pair creation.

Adding public key on Github.

We are also going to assume that you have installed the duckietown/software in ~/duckietown.

Compiling the documentation (updated Sep 12)

✎🔗

Make sure you have deployed and activated the virtual environment. You can check this by checking which python is active:

$ which python
/home/user/duckuments/deploy/bin/python

To compile the master versions of the docs, run:

$ make master-clean master

To see the result, open the file

./duckuments-dist/master/duckiebook/index.html

If you want to do incremental compilation, you can omit the clean and just use:

$ make master

This will be faster. However, sometimes it might get confused. At that point, do make master-clean.

Compiling the Fall 2017 version only (introduced Sep 12)

✎🔗

To compile the Fall 2017 versions of the docs, run:

$ make fall2017-clean fall2017

To see the result, open the file

./duckuments-dist/master/duckiebook/index.html

For incremental compilation, use:

$ make fall2017

Single-file compilation

✎🔗

There is also the option to compile one single file.

To do this, use:

$ ./compile-single path to .md file

This is the fastest way to see the results of the editing; however, there are limitations:

  • no links to other sections will work.
  • not all images might be found.

The workflow to edit documentation (updated Sep 12)

✎🔗

This is the basic workflow:

  1. Create a branch called yourname-branch in the duckuments repository.
  2. Edit the Markdown in the yourname-branch branch.
  3. Run make master to make sure it compiles.
  4. Commit the Markdown and push on the yourname-branch branch.
  5. Create a pull request.
  6. Tag the group duckietown/gardeners.

Create a pull request from the command-line using hub.

Basic Markduck guide

✎🔗

The Duckiebook is written in Markduck, a Markdown dialect.

It supports many features that make it possible to create publication-worthy materials.

Figures

✎🔗

For any element, adding an attribute called figure-id with value fig:figure ID or tab:table ID will create a figure that wraps the element.

For example:

<div figure-id="fig:figure ID">
    figure content
</div>

It will create HMTL of the form:

<div id='fig:code-wrap' class='generated-figure-wrap'>
    <figure id='fig:figure ID' class='generated-figure'>
        <div>
            figure content
        </div>
    </figure>
</div>

To add a caption, add an attribute figure-caption:

<div figure-id="fig:figure ID" figure-caption="This is my caption">
    figure content
</div>

Alternatively, you can put anywhere an element figcaption with ID figure id:caption:

<element figure-id="fig:figure ID">
    figure content
</element>

<figcaption id='fig:figure ID:caption'>
    This the caption figure.
</figcaption>

To refer to the figure, use an empty link:

Please see [](#fig:figure ID).

The code will put a reference to “Figure XX”.

Linking to documentation

✎🔗

Establishing names of headers

✎🔗

You give IDs to headers using the format:

### header title {#topic ID}

For example, for this subsection, we have used:

### Establishing names of headers {#establishing}

With this, we have given this header the ID “establishing”.

Special paragraphs and environments

✎🔗

Special paragraphs tags

✎🔗

The system supports parsing of some special paragraphs.

some of these might be redundant and will be eliminated. For now, I am documenting what is implemented.

Special paragraphs must be separated by a line

✎🔗

A special paragraph is marked by a special prefix. The list of special prefixes is given in the next section.

There must be an empty line before a special paragraph; this is because in Markdown a paragraph starts only after an empty line.

This is checked automatically, and the compilation will abort if the mistake is found.

For example, this is invalid:

See: this book
See: this other book

This is correct:

See: this book

See: this other book

Similarly, this is invalid:

Author: author
Maintainer: maintainer

and this is correct:

Author: author

Maintainer: maintainer

Other div environments

✎🔗

For these, note the rules:

  • You must include markdown="1".
  • There must be an empty line after the first div and before the closing /div.

Notes and questions

✎🔗

There are three environments: “comment”, “question”, “doubt”, that result in boxes that can be expanded by the user.

These are the one-paragraph forms:

Comment: this is a comment on one paragraph.
comment

this is a comment on one paragraph.

Question: this is a question on one paragraph.
question

this is a question on one paragraph.

Doubt: I have my doubts on one paragraph.
doubt

I have my doubts on one paragraph.

These are the multiple-paragraphs forms:

<div class='comment' markdown='1'>
A comment...

A second paragraph...
</div>
comment

A comment…

A second paragraph…

<div class='question' markdown='1'>
A question...

A second paragraph...
</div>
question

A question…

A second paragraph…

<div class='doubt' markdown='1'>
A question...

Should it not be:

    $ alternative command

A second paragraph...
</div>
doubt

A question…

Should it not be:

$ alternative command

A second paragraph…

Because of mathjax bug

Using LaTeX constructs in documentation

✎🔗

Working knowledge of LaTeX.

LaTeX equations

✎🔗

We can refer to equations, such as \eqref{eq:one}:

\begin{equation} 2a = a + a \label{eq:one}\tag{1} \end{equation}

This uses align and contains \eqref{eq:two} and \eqref{eq:three}.

\begin{align} a &= b \label{eq:two}\tag{2} \\ &= c \label{eq:three}\tag{3} \end{align}

We can refer to equations, such as \eqref{eq:one}:

\begin{equation}
    2a = a + a          \label{eq:one}
\end{equation}

This uses `align` and contains  \eqref{eq:two} and \eqref{eq:three}.

\begin{align}
    a &= b       \label{eq:two} \\
      &= c       \label{eq:three}
\end{align}

Note that referring to the equations is done using the syntax \eqref{eq:name}, rather than [](#eq:name).

Bibliography support

✎🔗

You need to have installed bibtex2html.

The system supports Bibtex files.

Place *.bib files anywhere in the directory.

Then you can refer to them using the syntax:

[](#bib:bibtex ID)

For example:

Please see [](#bib:siciliano07handbook).

Will result in:

Please see [3].

Embedding Latex in Figures through SVG

✎🔗

In order to compile the figures into PDFs you need to have Inkscape installed. Instructions to download and install Inkscape are here.

To embed latex in your figures, you can add it directly to a file and save it as filename.svg file and save anywhere in the /docs directory.

You can run:

$ make process-svg-figs

And the SVG file will be compiled into a PDF figure with the LaTeX commands properly interpreted.

You can then include the PDF file in a normal way (Section 2.5 - Figures) using filename.pdf as the filename in the <img> tag.

Image saved as svg
Image as PDF after processing
Embedding LaTeX in images

It can take a bit of work to get the positioning of the code to appear properly on the figure.

Because of mathjax bug

Advanced Markduck guide

✎🔗

Embedding videos

✎🔗

It is possible to embed Vimeo videos in the documentation.

Do not upload the videos to your personal Vimeo account; they must all be posted to the Duckietown Engineering account.

This is the syntax:

<dtvideo src="vimeo:vimeo ID"/>

For example, this code:

<div figure-id="fig:example-embed">
    <figcaption>Cool Duckietown by night</figcaption>
    <dtvideo src="vimeo:152825632"/>
</div>

produces this result:

The video is at https://vimeo.com/152825632.

Cool Duckietown by night

Depending on the output media, the result will change:

  • On the online book, the result is that a player is embedded.
  • On the e-book version, the result is that a thumbnail is produced, with a link to the video;
  • On the dead-tree version, a thumbnail is produced with a QR code linking to the video (TODO).

move-here tag

✎🔗

If a file contains the tag move-here, the fragment pointed by the src attribute is moved at the place of the tag.

This is used for autogenerated documentation.

Syntax:

# Node `node`

<move-here src='#package-node-autogenerated'/>

*Compiling the PDF version

✎🔗

This part describes how to compile the PDF version.

The dependencies below are harder to install. If you don’t manage to do it, then you only lose the ability to compile the PDF. You can do make compile to compile the HTML version, but you cannot do make compile-pdf.

Markduck troubleshooting

✎🔗

Common mistakes with Markdown

✎🔗

Here are some common mistakes encountered.

Not properly starting a list

✎🔗

There must be an empty line before the list starts.

This is correct:

I want to learn:

- robotics
- computer vision
- underwater basket weaving

This is incorrect:

I want to learn:
- robotics
- computer vision
- underwater basket weaving

and it will be rendered as follows:

I want to learn: - robotics - computer vision - underwater basket weaving

Because of mathjax bug

The Duckuments bot

✎🔗

This is an advanced section mainly for Liam.

Documentation style guide

✎🔗

This chapter describes the conventions for writing the technical documentation.

Learning in Duckietown

✎🔗

Jacopo

This chapter is a draft.

We do not refer to teachers and students

We refer to learners. Instructors are learners ahead in the learning curve in comparison to other learners

We don’t examinate

We assess competences

The learning feedback loop

✎🔗

A relevant contribution of modern educational theory is recognizing the importance of feedback in the learning process ().

We love feedback, as it stands at the foundation of control systems theory.

The learning loop.

Here, we provide definitions of the key elements in a learning feedback loop, and analogies to their control systems counterparts.

Intended Learning Outcomes

✎🔗

Intended Learning Outcome
An intended learning outcome is a desired, measurable, output of the learning process.

Intended learning outcomes are:

  • the starting point in the construction of a learning activity ([1]),

  • more effective when formulated with active verbs,

  • the equivalent of reference trajectories, or setpoints, in control systems. They represent the ideal output of the controlled system.

Students will understand robotics

Students each build a Duckiebot, implement software and produce demos

Learning Activities

✎🔗

Learning Activities
Methods chosen by the instructor to ease the learners achievement of the intended learning outcomes.

Active learning practices ([1]) have been shown to improve learning.

Instructor explains at the blackboard

Learners work in groups to achieve objectives

Learning activities are analogous to the the output of the controller (instructor), or input to the system (learners). They have to meet the requirements imposed by external constraints, not shown in

Assessment

✎🔗

Assessment
An assessment is a procedure to quantify the level of fulfillment of learning outcomes.

An assessment is analogous to a sensor in a control system loop. It measures the system’s output.

Examples of assessments include: quizzes, colloquia, produced documentation, homework, etc.

Knowledge, Skills and Competences

✎🔗

Here, we provide definitions for knowledge, skills and competences, in addition to describing their relationships (Figure 10.2).

The relationship between Knowledge, Skills and Competences.

Knowledge

✎🔗

Knowledge
Theoretical facts and information aimed at enabling understanding, and generating or improving skills.

Bayesian inference is handy piece of knowledge when doing robotics.

Practice

✎🔗

Practice
Practical procedures aimed at generating or improving skills, either directly or indirectly by improving knowledge.

Exercises and proofs can be used to practice different skills.

Skills

✎🔗

Skills
A proficiency, facility or dexterity that enables carrying out a function. Skills stem from knowledge, practice and/or aptitude. Skills can be clustered in cognitive, technical and interpersonal, respectively relating to ideas, things and people.

Analyzing tradeoffs between performances and constraints is a critical cognitive skill for robotics.

Python language is a useful technical skill in robotics.

Public speaking is a valuable interpersonal skill useful beyond robotics.

In Duckietown we formalize didactic indivisible units, or atoms, aimed at improving skills through knowledge and practice. Knowledge atoms are listed in XXX. We define as practice atoms:

add general reference to all learning atmos, folder atoms_30_learning_material

Exercise
An exercise is a practice atom aimed at improving technical skills. Exercises are listed in XXX.

Exercises are targeted to different “things” to which technical skills are related. They may be mathematical exercises aimed at practicing a method, or they may be coding exercises aimed at practicing resolutions of hardware implementation challenges.

Proof
A proof is a practice atom aimed at improving cognitive skills.

Deriving the Kalman filter equations helps practice the idea that there is no better approach to state estimation for linear time invariant systems, with “well behaved” measurement and process noises.

Competences

✎🔗

Competences
Set of skills and/or knowledge that leads to superior performance in carrying out a function. Competences must be measurable.

Competences are desirable intended learning outcomes, and typically address the how of the learning process.

Programming is a competence. It requires a skill, e.g., Python, and knowledge, e.g., Bayesian inference, to know what to code. Practice can help improve knowledge or hone skills.

Because of mathjax bug

Knowledge graph

✎🔗

This chapter describes something that is not implemented yet.

Markdown format for text-like atoms

✎🔗

For the text-like resources, they are described in Markdown files.

The name of the file does not matter.

All files are encoded in UTF-8.

Each file starts with a H1 header. The contents is the title.

The header has the following attributes:

  1. The ID. ({#ID})
  2. The status is given by an attribute status, which should be value of the values in Table 11.1.
  3. (Optional) The language is given by an attribute lang ({lang=en-US}).
  4. (Optional) The type is given by an attribute type ({type=demo}).

Here is an example of a header with all the attributes:

# Odometry calibration {#odometry-calibration lang=en-US type='text/theory' status=ready}

This first paragraph will be used as the "summary" for this text.
calibration.en.md

And this is how the Italian translation would look like:

# Calibrazione dell'odometria {#odometry-calibration lang=it type='text/theory' status=draft}

Questo paragrafo sarà usato come un sommario del testo.
calibration.it.md

Translations

✎🔗

This part is not implemented yet.

Operation manual - Duckiebot

✎🔗

In this section you will find information to obtain the necessary equipment for Duckietowns and different Duckiebot configurations.

Because of mathjax bug

Duckiebot configurations

✎🔗

nothing

Knowledge of Duckiebot configuration naming conventions, their components and functionalities.

After reviewing the configurations, you can proceed to purchasing the components, reading a description of the components, or assembling your chosen configuration.

We define different Duckiebot configurations depending on their time of use and hardware components. This is a good starting point if you are wondering what parts you should obtain to get started.

Branch configuration releases: Fall 2017

✎🔗

All branches release their hardware in two phases, namely a and b.

Soldering boards (DB17)

✎🔗

Shiying

Parts: Duckiebot DB17 parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd). The configurations are described in Unit C-1 - Duckiebot configurations. In particular:

Tools: Solderer

Experience: some novice-level experience with soldering.

Time: 30 minutes

Soldered DC Motor HAT

It is better to be safe than sorry. Soldering is a potentially hazardous activity. There is a fire hazard as well as the risk of inhaling toxic fumes. Stop a second and make sure you are addressing the safety standards for soldering when following these instructions. If you have never soldered before, seek advice.

Preparing the power cable (DB17)

✎🔗

In configuration DB17 we will need a cable to power the DC motor HAT from the battery. The keen observer might have noticed that such a cable was not included in the DB17 Duckiebot parts chapter. Here, we create this cable by splitting open any USB-A cable, identifying and stripping the power wires, and using them to power the DC motor HAT. If you are unsure about the definitions of the different Duckiebot configurations, read Unit C-1 - Duckiebot configurations.

It is important to note that these instructions are relevant only for assembling a DB17-wjdc configuration Duckiebot (or any subset of it). If you intend to build a DB17-l configuration Duckiebot, you can skip these instructions.

One male USB-A to anything cable.

A pair of scissors.

A multimeter (only if you are not purchasing the suggested components)

Time: 5 minutes

One male USB-A to wires power cable

Assembling the Duckiebot (DB17-jwd)

✎🔗

Shiying Li

Once you have received the parts and soldered the necessary components, it is time to assemble them in a Duckiebot. Here, we provide the assembly instructions for configurations DB17-wjd.

Duckiebot DB17-wjd parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd).

Having soldered the DB17-wjd parts. The soldering process is explained in Unit C-3 - Soldering boards (DB17).

Having prepared the power cable. The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17). Note: Not necessary if you intend to build a DB17-l configuration.

Having installed the image on the MicroSD card. The instructions on how to reproduce the Duckiebot system image are in Unit C-7 - Reproducing the image.

Time: about 40 minutes.

An assembled Duckiebot in configuration DB17-wjd.

The FAQ section at the bottom of this page may already answer some of you comments, questions or doubts.

While assembling the Duckiebot, try to make as symmetric (along the longitudinal axis) as you can. It will help going forward.

FAQ

✎🔗

If we have the bumpers, at what point should we add them?

You shouldn’t have the bumpers at this point. The function of bumpers is to keep the LEDs in place, i.e., they belong to DB17-l configuration. These instructions cover the DB17-jwd configurations. You will find the bumper assembly instructions in Unit E-3 - Assembling the Duckiebot (DB17-lc).

Yeah but I still have the bumpers and am reading this page. So?

The bumpers can be added after the Duckiebot assembly is complete.

I found it hard to mount the camera (the holes weren’t lining up).

Sometimes in life you have to push a little to make things happen. (But don’t push too much or things will break!)

The long camera cable is a bit annoying - I folded it and shoved it in between two hats.

The shorter cable is even more annoying. We suggest wrapping the long camera cable between the chassis and the Raspberry Pi. With some strategic planning, you can use the zipties that keep the battery in place to hold the camera cable in place as well (see figure below-to add)

add pretty cable handling pic

I found that the screwdriver that comes with the chassis kit is too fat to screw in the wires on the hat.

It is possible you got one of the fatter screwdrivers. You will need to figure it out yourself (or ask a TA for help).

I need something to cut the end of the zip tie with.

Scissors typically work out for these kind of jobs (and no, they’re not provided in a Fall 2017 Duckiebox).

Because of mathjax bug

Assembling the Duckiebot (DB17-wjd TTIC)

✎🔗

Andrea F. Daniele

Once you have received the parts and soldered the necessary components, it is time to assemble them in a Duckiebot. Here, we provide the assembly instructions for the configuration DB17-wjd (TTIC only).

Duckiebot DB17-wjd parts. The acquisition process is explained in Unit C-2 - Acquiring the parts (DB17-jwd).

Having soldered the DB17-wjd parts. The soldering process is explained in Unit C-3 - Soldering boards (DB17).

Having prepared the power cable. The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17). Note: Not necessary if you intend to build a DB17-l configuration.

Time: about 30 minutes.

An assembled Duckiebot in configuration DB17-wjd.

The FAQ section at the bottom of this page may already answer some of you comments, questions or doubts.

This section is comprised of 14 parts. Each part builds upon some of the previous parts, so make sure to follow them in the following order.

Motors

✎🔗

Open the Magician Chassis package (Figure 5.1) and take out the following components:

  • Chassis-bottom (1x)
  • DC Motors (2x)
  • Motor holders (4x)
  • M3x30 screw (4x)
  • M3 nuts (4x)

Figure 6.1 shows the components needed to complete this part of the tutorial.

Components needed to mount the motors.

Wheels

✎🔗

From the Magician Chassis package take the following components:

  • Wheels (2x)

Figure 6.7 shows the components needed to complete this part of the tutorial.

The wheels.

Omni-directional wheel

✎🔗

The Duckiebot is driven by controlling the wheels attached to the DC motors. Still, it requires a passive support on the back. In this configuration an omni-directional wheel is attached to the bottom plate of the chassis to provide such support.

From the Magician Chassis package take the following components:

  • Steel omni-directional wheel (1x)
  • Long metal spacers (2x)
  • M3x6 screws (4x)

Figure 6.10 shows the components needed to complete this part of the tutorial.

The omni-directional wheel with *2* long spacers and *4* M3x6 screws.

Chassis standoffs

✎🔗

From the Magician Chassis package take the following components:

  • Long metal spacers/standoffs (4x)
  • M3x6 screws (4x)

From the Duckiebot kit take the following components:

  • M3x5 nylon spacers/standoffs (4x)

Figure 6.14 shows the components needed to complete this part of the tutorial.

The standoffs to mount on the bottom plate.

Camera kit

✎🔗

From the Magician Chassis package take the following components:

  • M3x10 flathead screws (2x)
  • M3 nuts (2x)

From the Duckiebot kit take the following components:

  • Camera Module (1x)
  • (Optional) 300mm Camera cable (1x)
  • Camera mount (1x)

If you have camera cables of different lengths available, keep in mind that both are going to work. We suggest to use the longer one, and wrap the extra length under the Raspberry Pi stack.

Figure 6.18 shows the components needed to complete this part of the tutorial.

The parts needed to fix the camera on the top plate.

Heat sinks

✎🔗

From the Duckiebot kit take the following components:

  • Raspberry Pi 3 (1x)
  • Heat sinks (2x)
  • Camera mount (1x)

Figure 6.24 shows the components needed to complete this part of the tutorial.

The heat sinks and the Raspberry Pi 3.

Raspberry Pi 3

✎🔗

From the Magician Chassis package take the following components:

  • Top plate (with camera attached) (1x)

From the Duckiebot kit take the following components:

  • Raspberry Pi 3 (with heat sinks) (1x)
  • M2.5x12 nylon spacers/standoffs (8x)
  • M2.5 nylon hex nuts (4x)

Figure 6.27 shows the components needed to complete this part of the tutorial.

The parts needed to mount the Raspberry Pi 3 on the top plate.

Top plate

✎🔗

From the Magician Chassis package take the following components:

  • Bottom plate (with motors, wheels and standoffs attached) (1x)
  • Top plate (with camera and Raspberry Pi 3 attached) (1x)
  • M3x6 screws (4x)

Figure 6.32 shows the components needed to complete this part of the tutorial.

The parts needed to secure the top plate to the bottom plate.

USB Power cable

✎🔗

The power cable preparation is explained in Unit C-4 - Preparing the power cable (DB17).

DC Stepper Motor HAT

✎🔗

From the Duckiebot kit take the following components:

Figure 6.36 shows the components needed to complete this part of the tutorial.

The parts needed to add the DC Stepper Motor HAT to the Duckiebot.

Battery

✎🔗

From the Duckiebot kit take the following components:

  • Battery (1x)
  • Zip tie (1x)
  • Short micro USB cable (1x)

Figure 6.43 shows the components needed to complete this part of the tutorial.

The parts needed to add the battery to the Duckiebot.

Upgrade to DB17-w

✎🔗

This upgrade equips the Duckiebot with a secondary, faster, Wi-Fi connection, ideal for image streaming. The new configuration is called DB17-w.

Figure 6.46 shows the components needed to complete this upgrade.

The parts needed to upgrade the Duckiebot to the configuration DB17-w.

Upgrade to DB17-j

✎🔗

This upgrade equips the Duckiebot with manual remote control capabilities. It is particularly useful for getting the Duckiebot out of tight spots or letting younger ones have a drive, in addition to providing handy shortcuts to different functions in development phase. The new configuration is called DB17-j.

Figure 6.48 shows the components needed to complete this upgrade.

The parts needed to upgrade the Duckiebot to the configuration DB17-j.

The joystick comes with a USB receiver (as shown in Figure 6.48).

Upgrade to DB17-d

✎🔗

This upgrade equips the Duckiebot with an external hard drive that is convenient for storing videos (logs) as it provides both extra capacity and faster data transfer rates than the microSD card in the Raspberry Pi 3. Moreover, it is easy to unplug it from the Duckiebot at the end of the day and bring it over to a computer for downloading and analyzing stored data. The new configuration is called DB17-d.

Figure 6.50 shows the components needed to complete this upgrade.

The parts needed to upgrade the Duckiebot to the configuration DB17-d.

FAQ

✎🔗

If we have the bumpers, at what point should we add them?

You shouldn’t have the bumpers at this point. The function of the bumpers is to keep the LEDs in place, i.e., they belong to DB17-l configuration. These instructions cover the DB17-wjd configurations. You will find the bumper assembly instructions in Unit E-3 - Assembling the Duckiebot (DB17-lc).

Yeah but I still have the bumpers and am reading this page. So?

The bumpers can be added after the Duckiebot assembly is complete.

I found it hard to mount the camera (the holes weren’t lining up).

Sometimes in life you have to push a little to make things happen. (But don’t push too much or things will break!)

The long camera cable is a bit annoying - I folded it and shoved it in between two hats.

The shorter cable is even more annoying. We suggest wrapping the long camera cable between the chassis and the Raspberry Pi. With some strategic planning, you can use the zipties that keep the battery in place to hold the camera cable in place as well (see figure below-to add)

add pretty cable handling pic

I found that the screwdriver that comes with the chassis kit is too fat to screw in the wires on the hat.

It is possible you got one of the fatter screwdrivers. You will need to figure it out yourself (or ask a TA for help).

I need something to cut the end of the zip tie with.

Scissors typically work out for these kind of jobs (and no, they’re not provided in a Fall 2017 Duckiebox).

Because of mathjax bug

Reproducing the image

✎🔗

These are the instructions to reproduce the Ubuntu image that we use.

Please note that the image is already available, so you don’t need to do this.

However, this documentation is useful if you would like to port the software to a different distribution. Also, we periodically run through these instructions to make sure that they work.

Just in case, let’s re-state this: in Fall 2017, you don’t necessarily need to do the following.

Internet connection to download the packages.

A PC running any Linux with an SD card reader.

Time: about 4 hours (most of it spent waiting for things to download/compile).

A baseline Ubuntu Mate 16.04.2 image with updated software.

Passwordless sudo

✎🔗

First, make vi the default editor, using

$ sudo update-alternatives --config editor

and then choose vim.basic.

Then run:

$ sudo visudo

And then change this line:

%sudo   ALL=(ALL:ALL) ALL

into this line:

%sudo   ALL=(ALL:ALL) NOPASSWD:ALL

Installing Ubuntu on laptops

✎🔗

Andrea

Before you prepare the Duckiebot, you need to have a laptop with Ubuntu installed.

A laptop with free disk space.

Internet connection to download the Ubuntu image.

About 30 minutes.

A laptop ready to be used for Duckietown.

Duckiebot Initialization

✎🔗

Andrea

An SD card of dimensions at least 16 GB.

A computer with an internet connection, an SD card reader, and 16 GB of free space.

An assembled Duckiebot in configuration DB17. This is the result of Unit C-5 - Assembling the Duckiebot (DB17-jwd).

A Duckiebot that is configured correctly, that you can connect to with your laptop and hopefully also has internet access

Acquire and burn the image

✎🔗

On the laptop, download the compressed image at this URL:

https://www.dropbox.com/s/ckpqpp0cav3aucb/duckiebot-RPI3-AD-2017-09-12.img.xz?dl=1

The size is 1.7 GB.

You can use:

$ wget -O duckiebot-RPI3-AD-2017-09-12.img.xz URL above
comment

The original was:

$ curl -o duckiebot-RPI3-AD-2017-09-12.img.xz URL above

It looks like that curl cannot be used with Drobpox links because it does not follow redirects.

To make sure that the image is downloaded correctly, compute its hash using the program sha256sum:

$ sha256sum duckiebot-RPI3-AD-2017-09-12.img.xz
7136f9049b230de68e8b2d6df29ece844a3f830cc96014aaa92c6d3f247b6130  duckiebot-RPI3-AD-2017-09-12.img.xz

Compare the hash that you obtain with the hash above. If they are different, there was some problem in downloading the image.

Uncompress the file:

$ xz -d -k --verbose duckiebot-RPI3-AD-2017-09-12.img.xz

This will create a file of 11 GB in size.

Next, burn the image on disk.

The procedure of how to burn an image is explained in Section 17.2 - How to burn an image to an SD card.

Expand your filesystem

✎🔗

If your SD card is larger than the image, you’ll want to expand the filesystem on your robot so that you can use all of the space available. Achieve this with:

duckiebot $ sudo raspi-config --expand-rootfs

and then reboot

duckiebot $ sudo shutdown -r now

once rebooted you can test whether this was successful by doing

duckiebot $ df -lh

the output should give you something like:

Filesystem      Size  Used Avail Use% Mounted on
/dev/root        15G  6.3G  8.2G  44% /
devtmpfs        303M     0  303M   0% /dev
tmpfs           431M     0  431M   0% /dev/shm
tmpfs           431M   12M  420M   3% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           431M     0  431M   0% /sys/fs/cgroup
/dev/mmcblk0p1   63M   21M   43M  34% /boot
tmpfs            87M     0   87M   0% /run/user/1000

You should see that the Size of your /dev/root Filesystem is “close” to the size of your SD card.

Create your user

✎🔗

You must not use the ubuntu user for development. Instead, you need to create a new user.

Choose a user name, which we will refer to as username.

To create a new user:

duckiebot $ sudo useradd -m username

Make the user an administrator by adding it to the group sudo:

duckiebot $ sudo adduser username sudo

Make the user a member of the groups input, video, and i2c

duckiebot $ sudo adduser username input
duckiebot $ sudo adduser username video
duckiebot $ sudo adduser username i2c

Set the shell bash:

duckiebot $ sudo chsh -s /bin/bash username

To set a password, use:

duckiebot $ sudo passwd username

At this point, you should be able to login to the new user from the laptop using the password:

laptop $ ssh username@robot name

Next, you should repeat some steps that we already described.

comment

What steps?? -LP

Add SSH alias

✎🔗

Once you have your SSH key pair on both your laptop and your Duckiebot, as well as your new user- and hostname set up on your Duckiebot, then you should set up an SSH alias as described in Section 15.1 - SSH aliases. This allows your to log in for example with

laptop $ ssh abc

instead of

laptop $ ssh username@robot name

where you can chose abc to be any alias / shortcut.

Hardware check: camera

✎🔗

Check that the camera is connected using this command:

duckiebot $ vcgencmd get_camera
supported=1 detected=1

If you see detected=0, it means that the hardware connection is not working.

You can test the camera right away using a command-line utility called raspistill.

Use the raspistill command to capture the file out.jpg:

duckiebot $ raspistill -t 1 -o out.jpg

Then download out.jpg to your computer using scp for inspection.

For instructions on how to use scp, see Subsection 21.1.1 - Download a file with SCP.

✎🔗

In order to show that your Duckiebot is ready for the task of driving around happy little duckies, the robot has to fly the Duckietown flag. When you are still logged in to the Duckiebot you can download and install the banner like this:

Download the ANSI art file from Github:

duckiebot $ wget --no-check-certificate -O duckie.art "https://raw.githubusercontent.com/duckietown/Software/master/misc/duckie.art"

(optional) If you want, you can preview the logo by just outputting it onto the command line:

duckiebot $ cat duckie.art

Next up create a new empty text file in your favorite editor and add the code for showing your duckie pride:

Let’s say I use nano, I open a new file:

duckiebot $ nano 20-duckie

And in there I add the following code (which by itself just prints the duckie logo):

#!/bin/sh
printf "\n$(cat /etc/update-motd.d/duckie.art)\n"

Then save and close the file. Finally you have to make this file executable…

duckiebot $ chmod +x 20-duckie

…and copy both the duckie logo and the script into a specific directory /etc/update-motd.d to make it appear when you login via SSH. motd stands for “message of the day”. This is a mechanism for system administrators to show users news and messages when they login. Every executable script in this directory which has a filename a la NN-some name will get exected when a user logs in, where NN is a two digit number that indicates the order.

sudo cp duckie.art /etc/update-motd.d
sudo cp 20-duckie /etc/update-motd.d

Finally log out of SSH via exit and log back in to see duckie goodness.

Networking aka the hardest part

✎🔗

A Duckiebot in configuration DB17-CO+w

Either a router that you have control over that has internet access, or your credentials for connecting to an existing wireless network

Patience (channel your inner Yoda)

A Duckiebot that you can connect to and that is connected to the internet

this page is primarily for folks operating with the “two-network” configuration, C0+w. For a one adapter setup you will can skip directly to Section 10.2 - Setting up wireless network configuration, but you will have to connect to a network that you can ssh through.

The basic idea is that we are going to use the “Edimax” thumbdrive adapter to create a dedicated wireless network that you can always connect to with your laptop. Then we are going to use the built-in Broadcom chip on the Pi to connect to the internet, and then the network will be bridged.

Setting up wireless network configuration

✎🔗

You are connected to the Duckiebot via WiFi, but the Duckiebot also needs to connect to the internet in order to get updates and install some software. This part is a little bit more of a “black art” since we cannot predict every possible network configurations. Below are some settings that have been verified to work in different situations:

Option 5: ETH Wifi

✎🔗

The following instructions will lead you to connect your PI to the “eth” wifi network.

First, run the following on duckiebot

duckiebot $ iwconfig
...

lo        no wireless extensions.

enxbxxxxxxxxxxx  no wireless extensions.

...

Make note of the name enxbxxxxxxxxxxx. xxxxxxxxxxx should be a string that has 11 characters that is formed by numbers and lower case letters.

Second, edit the file /etc/network/interfaces which requires sudo so that it looks like the following, and make sure the enxbxxxxxxxxxxx matches.

Pay special attention on the line “pre-up wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf”.This is expected to be exactly one line instead of two but due to formatting issue it is shown as two lines.

Also, make sure every characters match exactly with the provided ones. TAs will not help you to do spelling error check. 

# interfaces(5) file used by ifup(8) and ifdown(8) Include files from /etc/network/     interfaces.d:
source-directory /etc/network/interfaces.d

# The loopback network interface
auto lo
auto enxbxxxxxxxxxxx

# the wired network setting 
iface enxbxxxxxxxxxxx inet dhcp

# the wireless network setting
auto wlan0
allow-hotplug wlan0
iface wlan0 inet dhcp
    pre-up wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf
    post-down killall -q wpa_supplicant

Third, edit the file /etc/wpa_supplicant/wpa_supplicant.conf which requires sudo so that it looks like the following, and make sure you substitute [identity] and [password] content with your eth account information: 

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="eth"
    key_mgmt=WPA-EAP
    group=CCMP TKIP
    pairwise=CCMP TKIP
    eap=PEAP
    proto=RSN
    identity="your user name goes here"
    password="your password goes here"
    phase1="peaplabel=0"
    phase2="auth=MSCHAPV2"
    priority=1
}

Fourth, reboot your PI. 

duckiebot $ sudo reboot

Then everything shall be fine. The PI will connect to “eth” automatically everytime it starts.

Because of mathjax bug

Software setup and RC remote control

✎🔗

Laptop configured, according to Unit C-8 - Installing Ubuntu on laptops.

You have configured the Duckiebot. The procedure is documented in Unit C-9 - Duckiebot Initialization.

You have created a Github account and configured public keys, both for the laptop and for the Duckiebot. The procedure is documented in Unit J-29 - Setup Github access.

You can run the joystick demo.

Clone the Duckietown repository

✎🔗

Clone the repository in the directory ~/duckietown:

duckiebot $ git clone git@github.com:duckietown/Software.git ~/duckietown

For the above to succeed you should have a Github account already set up.

It should not ask for a password.

you must not clone the repository using the URL starting with https. Later steps will fail.

Set up the ROS environment on the Duckiebot

✎🔗

All the following commands should be run in the ~/duckietown directory:

duckiebot $ cd ~/duckietown

Now we are ready to make the workspace. First you need to source the baseline ROS environment:

duckiebot $ source /opt/ros/kinetic/setup.bash

Then, build the workspace using:

duckiebot $ catkin_make -C catkin_ws/

For more information about catkin_make, see Section 30.6 - catkin_make.

there is a known bug, for which it fails the first time on the Raspberry Pi. Try again; it will work.

comment

I got no error on first execution on the Raspberry Pi

Clone the duckiefleet repository

✎🔗

Clone the relevant duckiefleet repository into ~/duckiefleet.

See see Subsection 4.1.2 - Duckiefleet directory DUCKIEFLEET_ROOT to find the right duckiefleet repository.

In ~/.bashrc set DUCKIEFLEET_ROOT to point to the directory:

export DUCKIEFLEET_ROOT=~/duckiefleet

Also, make sure that you execute ~/.bashrc in the current shell by running the command:

source ~/.bashrc

Add your vehicle data to the robot database

✎🔗

Next, you need to add your robot to the vehicles database. This is not optional and required in order to launch any ROS scripts.

You have already a copy of the vehicles database in the folder robots of DUCKIEFLEET_ROOT.

Copy the file emma.robot.yaml to robotname.robot.yaml, where robotname is your robot’s hostname. Then edit the copied file to represent your Duckiebot.

For information about the format, see Section 4.2 - The “scuderia” (vehicle database).

Generate the machines file.

The procedure is listed here: Section 4.3 - The machines file.

Finally, push your robot configuration to the duckiefleet repo.

Test that the joystick is detected

✎🔗

Plug the joystick receiver in one of the USB port on the Raspberry Pi.

To make sure that the joystick is detected, run:

duckiebot $ ls /dev/input/

and check if there is a device called js0 on the list.

Make sure that your user is in the group input and i2c:

duckiebot $ groups
username sudo input i2c

If input and i2c are not in the list, you missed a step. Ohi ohi! You are not following the instructions carefully!

Consult again Section 9.11 - Create your user.

To test whether or not the joystick itself is working properly, run:

duckiebot $ jstest /dev/input/js0

Move the joysticks and push the buttons. You should see the data displayed change according to your actions.

Reading from the camera

✎🔗

You have configured the Duckiebot. The procedure is documented in Unit C-9 - Duckiebot Initialization.

You know the basics of ROS (launch files, roslaunch, topics, rostopic).

put reference

You know that the camera works under ROS.

RC control launched remotely

✎🔗

Andrea

You can run the joystick demo from the Raspberry Pi. The procedure is documented in Unit C-11 - Software setup and RC remote control.

You can run the joystick demo from your laptop.

Rebuild the machines files

✎🔗

In a previous step you have created a robot configuration file and pushed it to the duckiefleet repo. Now you have to pull duckiefleet on the laptop and rebuild the machines configuration file there.

The procedure is documented in Section 11.5 - Add your vehicle data to the robot database.

Start the demo

✎🔗

Now you are ready to launch the joystick demo remotely.

Make sure that you can login with SSH without a password. From the laptop, run:

laptop $ ssh username@robot name.local

If this doesn’t work, you missed some previous steps.

Run this on the laptop:

laptop $ source environment.sh
laptop $ roslaunch duckietown joystick.launch veh:=robot name

You should be able to drive the vehicle with joystick just like the last example. Note that remotely launching nodes from your laptop doesn’t mean that the nodes are running on your laptop. They are still running on the Raspberry Pi in this case.

For more information about roslaunch, see Section 30.3 - roslaunch.

RC+camera remotely

✎🔗

Andrea

You can run the joystick demo remotely. The procedure is documented in Unit C-13 - RC control launched remotely.

You can read the camera data from ROS. The procedure is documented in Unit C-12 - Reading from the camera.

You know how to get around in Byobu. You can find the Byobu tutorial in Unit J-26 - Byobu.

You can run the joystick demo from your laptop and see the camera image on the laptop.

Interlude: Ergonomics

✎🔗

Andrea

So far, we have been spelling out all commands for you, to make sure that you understand what is going on.

Now, we will tell you about some shortcuts that you can use to save some time.

in the future you will have to debug problems, and these problems might be harder to understand if you rely blindly on the shortcuts.

Time: 5 minutes.

You will know about some useful shortcuts.

SSH aliases

✎🔗

Instead of using

$ ssh username@robot name.local

You can set up SSH so that you can use:

$ ssh my-robot

To do this, create a host section in ~/.ssh/config on your laptop with the following contents:

Host my-robot
    User username
    Hostname robot name.local

Here, you can choose any other string in place of “my-robot”.

Note that you cannot do

$ ping my-robot

You haven’t created another hostname, just an alias for SSH.

However, you can use the alias with all the tools that rely on SSH, including rsync and scp.

Wheel calibration

✎🔗

Andrea Daniele

You can run the joystick demo remotely. The procedure is documented in Unit C-13 - RC control launched remotely.

Calibrate the wheels of the Duckiebot such that it goes in a straight line when you command it to. Set the maximum speed of the Duckiebot.

Camera calibration

✎🔗

You can see the camera image on the laptop. The procedure is documented in Unit C-14 - RC+camera remotely.

You have all the repositories (described in Unit M-7 - Git usage guide for Fall 2017) cloned properly and you have your environment variables set properly.

Calibration for the robot camera.

Taking and verifying a log

✎🔗

Unit C-12 - Reading from the camera

Unit C-11 - Software setup and RC remote control

A verified log.

Record the log

✎🔗

Verify a log

✎🔗

On the Duckiebot run:

duckiebot $ rosbag info FULL_PATH_TO_BAG --freq

Then:

  • verify that the “duration” of the log seems “reasonable” - it’s about as long as you ran the log command for

  • verify that the “size” of the log seems “reasonable” - the log size should grow at about 220MB/min

  • verify in the output that your camera was publishing very close to 30.0Hz and verify that you joysick was publishing at a rate between 3Hz and 6Hz.

More complex log verification methods.

Because of mathjax bug

Troubleshooting

✎🔗

The Raspberry Pi of the Duckiebot is connected to the battery.

The Stepper Motor HAT is connected to the battery.

You have a problem!

I cannot access my Duckiebot via SSH

✎🔗

When I run ssh robot_name.local I get the error ssh: Could not resolve hostname robot_name.local.

Make sure that your Duckiebot is ON. Connect it to a monitor, a mouse and a keyboard. Run the command

duckiebot $ sudo service avahi-daemon status

You should get something like the following

● avahi-daemon.service - Avahi mDNS/DNS-SD Stack
   Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2017-10-22 00:07:53 CEST; 1 day 3h ago
 Main PID: 699 (avahi-daemon)
   Status: "avahi-daemon 0.6.32-rc starting up."
   CGroup: /system.slice/avahi-daemon.service
           ├─699 avahi-daemon: running [robot_name_in_avahi.local
           └─727 avahi-daemon: chroot helpe

Avahi is the module that in Ubuntu implements the mDNS responder. The mDNS responder is responsible for advertising the hostname of the Duckiebot on the network so that everybody else within the same network can run the command ping robot_name.local and reach your Duckiebot. Focus on the line containing the hostname published by the avahi-daemon on the network (i.e., the line that contains robot_name_in_avahi.local). If robot_name_in_avahi matches the robot_name, go to the next Resolution point. If robot_name_in_avahi has the form robot_name-XX, where XX can be any number, modify the file /etc/avahi/avahi-daemon.conf as shown below.

Identify the line

use-ipv6=yes

and change it to

use-ipv6=no

Identify the line

#publish-aaaa-on-ipv4=yes

and change it to

publish-aaaa-on-ipv4=no

Restart Avahi by running the command

duckiebot $ sudo service avahi-daemon restart

The Duckiebot does not move

✎🔗

I can SSH into my Duckiebot and run the joystick demo but the joystick does not move the wheels.

Press the button on the side of the battery (Figure 19.1).

Check that the red indicator on the joystick stopped blinking.

Bad joystick status

Bad joystick status

The joystick is connected (as shown in Figure 19.2b - Bad joystick status) but the Duckiebot still does not move.

Make sure that the controller is connected to the Duckiebot and that the OS receives the data from it. Run

duckiebot $ jstest /dev/input/js0

If you receive the error

jstest: No such file or directory

it means that the USB receiver is not connected to the Raspberry Pi or is broken. If the command above shows something like the following

Driver version is 2.1.0.
Joystick (ShanWan PC/PS3/Android) has 8 axes (X, Y, Z, Rz, Gas, Brake, Hat0X, Hat0Y)
and 15 buttons (BtnX, BtnY, BtnZ, BtnTL, BtnTR, BtnTL2, BtnTR2, BtnSelect, BtnStart, BtnMode, BtnThumbL, BtnThumbR, ?, ?, ?).
Testing ... (interrupt to exit)
Axes:  0:     0  1:     0  2:     0  3:     0  4:-32767  5:-32767  6:     0  7:     0 Buttons:  0:off  1:off  2:off  3:off  4:off  5:off  6:off  7:off  8:off  9:off 10:off 11:off 12:off 13:off 14:off

it means that the USB receiver is connected to the Raspberry Pi. Leave the terminal above open and use the joystick to command the Duckiebot. If you observe that the numbers shown in the terminal change according to the commands sent through the joystick than the problem is in ROS. Make sure that the joystick demo is launched. Restart the Duckiebot if needed and try again.

If the numbers do not change while using the joystick then follow this guide at the next Resolution point.

The controller might be connected to another Duckiebot nearby. Turn off the controller, go to a room with no other Duckiebots around and turn the controller back on. Retry.

Because of mathjax bug

Operation manual - Duckietown

✎🔗
Because of mathjax bug

Duckietown parts

✎🔗

Duckietowns are the cities where Duckiebots drive. Here, we provide a link to all bits and pieces that are needed to build a Duckietown, along with their price tag. Note that while the topography of the map is highly customable, we recommend using the components listed below. Before purchasing components for a Duckietown, read Unit D-2 - Duckietown Appearance Specification to understand how Duckietowns are built.

In general, keep in mind that:

  • The links might expire, or the prices might vary.
  • Shipping times and fees vary, and are not included in the prices shown below.
  • Substitutions are probably not OK, unless you are OK in writing some software.

Cost (per $m^2$): USD ??? + Shipping Fees

Time: ??? days (average shipping time)

A kit of parts ready to be assembled in a Duckietown.

Assembling a Duckietown.

Figure out costs

Traffic lights Parts

✎🔗

Traffic lights regulate intersections in Duckietown. Here, we provide a link to all bits and pieces that are needed to build a traffic light, along with their price tag. You will need one traffic per either three, or four way intersections. The components listed below meet the appearence specifications described in Unit D-2 - Duckietown Appearance Specification.

In general, keep in mind that:

  • The links might expire, or the prices might vary.
  • Shipping times and fees vary, and are not included in the prices shown below.
  • Substitutions are probably OK, if you are willing to write some software.

Cost: USD ??? + Shipping Fees

Time: ??? days (average shipping time)

A kit of parts ready to be assembled in a traffic light.

Assembling a traffic light.

Estimate time and costs

Duckietown Appearance Specification

✎🔗

Liam

This document describes the Duckietown specification. Any Duckietown not adhering to the rules described here cannot call itself a “Duckietown”, since it is not one. Additionally, any Duckietown not adhering to these rules may cause the Duckiebots to fail in unexpected ways. These are a set of rules for which a functional system has been verified.

Traffic Signs

✎🔗

To print and assemble the signs refer to Unit D-3 - Signage.

Placement

✎🔗

Signs may appear on the opposite side and at the corner of the adjacent tile from which they are viewed. In the absence of any signs, it is assumed that all network flows are allowed so a sign MUST be placed and visible whenever this is not the case.

Signs must only be placed on empty tiles, or next to one of the other tile types if on the border of a map. The sign placements for four different cases are shown in Figure 2.5. At intersections, from each stop line 2 signs should be clearly visible: 1) the intersection type (traffic light or stop sign) and 2) the intersection topology.

At present, 4-way intersections must be equipped with traffic lights for safe navigation.

4-way intersection
3-way intersection
straight road
curved road
Placement of Traffic Signs

On straight and curved roads, additional signs can be added as desired. Their placement is indicated in Figure 2.5c - straight road and Figure 2.5d - curved road. The signs should be placed at the border between two tiles and should face towards oncoming traffic as indicated.

In these figures the arrow is the direction of the sign.

Street Name Signs

✎🔗

Signage

✎🔗

Liam

The raw materials as described in Unit D-1 - Duckietown parts

A set of signs to be used for assembling your Duckietown.

Making New Signage

✎🔗

If you find that what is available in the database in insufficient for your needs, then you will need to add to the existing database.

To do so you will have to load the original AprilTags file available here: pdf ps

Which tag you should use depends on what type of sign you are trying add. The ranges of tags are specified in Table 3.1.

April tag ID ranges
Purpose Size Family ID Range
Traffic signs 6.5cm x 6.5cm 36h11 1-199
Traffic lights 6.5cm x 6.5cm 36h11 200-299
Localization 6.5cm x 6.5cm 36h11 300-399
Street Name Signs 6.5cm x 6.5cm 36h11 400-587

First, find the last sign of the type that you are trying to make in the signs and tags doc. You will use the next available ID after this one.

Construct the new sign by first copying and pasting an existing sign of similar type, and then replacing/adding the new AprilTag. To add the new april tag, use a screen capture mathod to crop precisely around the tag at the top and sides and include the sign id at the bottom. Then paste the tag into your word file under your desired and resize it exactly 6.5cm (2.56inches).

If you make a new road name sign, you may need to change the font size of the name so that it appears on one line (this is why we like people with names like “ROY” and “RUS”).

Important: You must also add your new sign to the April Tags DB in the software repo.

Add a new block like the ones that already exists or modify the one with the appropriate tag id:

- tag_id: NEW_TAG_ID
  tag_type: in {TrafficSign, Light, Localization, StreetName}
  street_name: either NEW_STREET_NAME or blank
  vehicle_name: currently not used
  traffic_sign_type: either TRAFFIC_SIGN_TYPE or blank

The value of NEW_STREET_NAME is up to you to decide (have fun with it!). The value of TRAFFIC_SIGN_TYPE should be one of the signs in Figure 2.4

When finished, regenerate the PDF version of the Word file, and commit everything to the repo (via a pull request of course).

It is also possible of course to start you own completely different signs and tags database, but make sure that you specify in the april_tags code which database to load from.

Update the way that the april tags code loads the database

Duckietown Assembly

✎🔗

Shiying

Follow the rules in the Unit D-2 - Duckietown Appearance Specification.

comment

do we need this page at all?

Because of mathjax bug

Traffic lights Assembly

✎🔗

Marco Erni

Because of mathjax bug

Semantics of LEDS

✎🔗

???

headlights: white, constant

Assumption:

  • 20 fps to do LED detection

  • 1s to decide

  • 3 frequencies to detect

tail lights: red, 6 hz square wave

traffic light “GO” = green, 1 hz square wave

traffic light “STOP” = red, 1.5 Hz square wave

duckie light on top, state 0 = off

duckie light on top, state 1 = blue, 3 Hz, square wave

duckie light on top, state 2 = ?, 2.5 Hz square wave

duckie light on top, state 3 = ?, 2 Hz square wave

Because of mathjax bug

Duckiebot - DB17-lc configurations

✎🔗

This section contains the acquisition, assembly and setup instructions for the DB17-lc configurations. These instructions are separate from the rest as they approximately match the b releases of the Fall 2017 Duckietown Engineering Co. branches.

To understand how configurations and releases are defined, refer to: Unit C-1 - Duckiebot configurations.

Because of mathjax bug

Acquiring the parts (DB17-lc)

✎🔗

Upgrading your DB17 (DB17-wjd) configuration to DB17-lc (DB17-wjdlc) starts here, with purchasing the necessary components. We provide a link to all bits and pieces that are needed to build a DB17-lc Duckiebot, along with their price tag. If you are wondering what is the difference between different Duckiebot configurations, read Unit C-1 - Duckiebot configurations.

In general, keep in mind that:

  • The links might expire, or the prices might vary.
  • Shipping times and fees vary, and are not included in the prices shown below.
  • Buying the parts for more than one Duckiebot makes each one cheaper than buying only one.
  • A few components in this configuration are custom designed, and might be trickier to obtain.

A Duckiebot in DB17-wjd configuration.

Cost: USD 77 + Bumpers manufacturing solution

Time: 21 Days (LED board manufacturing and shipping time)

A kit of parts ready to be assembled in a DB17-lc configuration Duckiebot.

After receiving these components, you are ready to do some soldering before assembling your DB17-lc Duckiebot.

Soldering boards (DB17-l)

✎🔗

General rule in soldering

  • soldering the components according to the height of components - from lowest to highest

Shiying

Duckiebot DB17-l parts. The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc). The configurations are described in Unit C-1 - Duckiebot configurations.

Time: 30 minutes

A DB17-l Duckiebot

Assembling the Duckiebot (DB17-lc)

✎🔗

Shiying

Duckiebot DB17-lc parts. The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc).

Soldering DB17-lc parts. The soldering process is explained in Unit E-2 - Soldering boards (DB17-l).

Having assembled the Duckiebot in configuration DB17 (or any DB17-wjd). The assembly process is explained in Unit C-5 - Assembling the Duckiebot (DB17-jwd).

Time: about 30 minutes.

An assembled Duckiebot in configuration DB17-wjdlc.

Bumper Assembly

✎🔗

Duckiebot DB17-lc parts.

Having the Duckiebot with configuration DB17-wjd assembled. The assembly process is explained in Unit E-3 - Assembling the Duckiebot (DB17-lc).

Time: about 15 minutes.

A Duckiebot with Bumpers (configuration DB17-l2)

DB17-l setup

✎🔗

Shiying

Duckiebot DB17-lc parts. The acquisition process is explained in Unit E-1 - Acquiring the parts (DB17-lc).

Soldering DB17-lc parts. The soldering process is explained in Unit E-2 - Soldering boards (DB17-l).

Having assembled PWM Hat on the Duckiebot with configuration DB17-wjd. The assembly process is explained in Unit E-3 - Assembling the Duckiebot (DB17-lc).

Time: about 15 minutes.

A Duckiebot with LEDs attached (configuration DB17-l3)

Operation Manual - demos

✎🔗

This part describes a set of demos that demonstrate the functionality of the robots.

Some of these demos require different hardware on the robot, or in Duckietown.

Because of mathjax bug

Demo template

✎🔗

This is the template for the description of a demo.

First, we describe what is needed, including:

  • Robot hardware
  • Number of Duckiebots
  • Robot setup steps
  • Duckietown hardware

Duckiebot in configuration ???

Camera calibration completed.

Video of expected results

✎🔗

First, we show a video of the expected behavior (if the demo is succesful).

Duckietown setup notes

✎🔗

Here, describe the assumptions about the Duckietown, including:

  • Layout (tiles types)
  • Instrastructure (traffic lights, wifi networks, …) required
  • Weather (lights, …)

Do not write instructions here. The instructions should be somewhere in the part about Duckietowns. Here, merely point to them.

Duckiebot setup notes

✎🔗

Write here any special setup for the Duckiebot, if needed.

Do not write instructions here. The instructions should be somewhere in the appropriate setup part.

Pre-flight checklist

✎🔗

The pre-flight checklist describes the steps that are sufficient to ensure that the demo will be correct:

Check: operation 1 done

Check: operation 2 done

Demo instructions

✎🔗

Here, give step by step instructions to reproduce the demo.

Step 1: XXX

Step 2: XXX

Troubleshooting

✎🔗

Add here any troubleshooting / tips and tricks required.

Demo failure demonstration

✎🔗

Finally, put here a video of how the demo can fail, when the assumptions are not respected.

Because of mathjax bug

Lane following

✎🔗

This is the description of lane following demo.

Wheels calibration completed.wheel calibration

Camera calibration completed.Camera calibration

Joystick demo has been successfully launched.Joystick demo

Video of expected results

✎🔗

link 1 of lane following link 2 of lane following

upload videos to vimeo and embed them here.

Duckietown setup notes

✎🔗

A duckietown with white and yellow lanes. No obstacles on the lane.

Duckiebot setup notes

✎🔗

Make sure the camera is heading ahead. Tighten the screws if necessary.

Pre-flight checklist

✎🔗

Check: turn on joystick.

Check: Enough battery of the duckiebot.

Demo instructions

✎🔗

Here, give step by step instructions to reproduce the demo.

Step 1: On duckiebot, in /DUCKIERTOWN_ROOT/ directory, run command:

duckiebot $ make demo-lane-following

Wait a while so that everything has been launched. Press R1 to start autonomous lane following. Press L1 to switch to joystick control. Press X to start anti-instagram. Empirically speaking, no duckiebot will successfully run the demo for its first time. Parameter tuning is a must. The only two parameters that you can modify is the gain and trim. The parameter pair which makes your bot go straight will unlikely work for the lane following due to the current controller design. Facts show that a gain ranging from 0.5 to 0.9, as long as paired with a suitable trim, will all work on this demo. Start with your parameter pair obtained from wheel calibration. Increase gain for higher speed. Increase trim to horizontally move the bot to the center of the lane. Decrease will do the inverse.

Step 2: On laptop, make sure ros enviroment has been activated, run command:

laptop $ rviz

In rviz, two markerarrays /(vehicle_name)/duckiebot_visualizer/segment_list_markers and /(vehicle_name)/lane_pose_visualizer_node/lane_pose_markers can be visualized. The green arrow shall be in a reasonable direction.

Step 3: On laptop, make sure ros enviroment has been activated, run command:

laptop $ rqt

In rqt, the images can be visualized are /(vehicle_name)/camera_node/image/compressed, /(vehicle_name)/line_detector_node/image_with_lines, /(vehicle_name)/lane_filter_node/belief_img.

Troubleshooting

✎🔗

Contact Yang Shaohui(ETHZ) via Slack if any trouble occurs.

Because of mathjax bug

Intersection Navigation

✎🔗

???

Because of mathjax bug

Indefinite Navigation

✎🔗

This is the description of the indefinite navigation demo.

Wheels calibration completed.wheel calibration

Camera calibration completed.Camera calibration

Joystick demo has been successfully launched.Joystick demo

Fully set up Duckietown (including April tags for intersections)

A motor gain of approximately 0.65 (strong influence in open-loop intersections)

Video of expected results

✎🔗
Resource error
Cannot open URL

   http://vimeo.com/api/v2/video/247596730.json
| Traceback (most recent call last):
|   File "/home/duckietown/scm/duckuments/mcdp/src/mcdp_docs/videos.py", line 110, in get_vimeo_info
|     response = urllib2.urlopen(url)
|   File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
|     return opener.open(url, data, timeout)
|   File "/usr/lib/python2.7/urllib2.py", line 435, in open
|     response = meth(req, response)
|   File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
|     'http', request, response, code, msg, hdrs)
|   File "/usr/lib/python2.7/urllib2.py", line 473, in error
|     return self._call_chain(*args)
|   File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
|     result = func(*args)
|   File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
|     raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
| HTTPError: HTTP Error 503: Service Unavailable
Demo: indefinite navigation

add a different video with an up to specification Duckietown.

Duckietown setup notes

✎🔗

A Duckietown with white and yellow lanes. No obstacles on the lane. Red stop lines at intersections. If several Duckiebots are present while running this demo, LEDs need to be installed for explicit communication and coordination of Duckiebots at intersections.

Duckiebot setup notes

✎🔗

Make sure the camera is securely tightened and properly calibrated.

Pre-flight checklist

✎🔗

Check: Joystick is turned on.

Check: Sufficient battery charge of the Duckiebot.

Check: Gain is set to approximately 0.65.

Demo instructions

✎🔗

Follow these steps to run the indefinite navigation demo on your Duckiebot:

Step 1: On the Duckiebot, navigate to the /DUCKIETOWN_ROOT/ directory, run the command:

duckiebot $ make indefinite-navigation

Wait until everything has been launched. Press X to start anti-instagram. Pressing R1 will start the autonomous lane following and L1 can be used to revert back to manual control.

In the current open-loop intersection navigation, no Duckiebot will successfully run the demo the first time. Parameter tuning is a must. The only two parameters that should be adjusted are the gain and trim, previously defined during the wheel calibration procedure.

The parameter pair which makes your bot go straight will unlikely work for the lane following due to the current controller design. Start with your parameter pair obtained from wheel calibration. If your Duckiebot stays too long on a curve during crossing an intersection, decrease your gain in steps of 0.05. If the Duckiebot doesn’t make the turn enough long, increase your gain in steps of 0.05.

Command to modify your gain (in this example to 0.65):

$ rosservice call /robot name/inverse_kinematics_node/set_gain -- 0.65
question

on laptop or bot?

Everything below is helpful for debugging if your robot does not follow the lane at all.

Step 2: Navigate to the Duckietown folder:

laptop $ cd ~/duckietown

then source the environment:

laptop $ source environment.sh

set the the ROS master to your vehicle:

laptop $ source set_ros_master.sh robot name

and finally launch rviz:

laptop $ rviz

In rviz, two markerarrays:

  • /robot name/duckiebot_visualizer/segment_list_markers, and
  • /robot name/lane_pose_visualizer_node/lane_pose_markers

can be visualized. The green arrow representing the pose estimate of the robot has to be in a reasonable direction.

Step 3: Always on the laptop, run:

laptop $ rqt

In rqt, the images can be visualized are:

  • /robot name/camera_node/image/compressed,
  • /robot name/line_detector_node/image_with_lines,
  • /robot name/lane_filter_node/belief_img.

Troubleshooting

✎🔗

Contact Julien Kindle(ETHZ) via Slack for further assistance.

Because of mathjax bug

Coordination

✎🔗

This demo (instructions from MIT2016) allow multiple Duckiebots that stop at an intersection to coordinate with LED signals and clear the intersection. Two demos are available, one for the case with traffic lights and one without them.

Duckiebot in configuration DB17-lc

Camera calibration and kinematic calibration completed.

Demo instructions

✎🔗

Troubleshooting

✎🔗
  • When it shows “Event:intersection_go”, the Duckiebot does not move. This problem is related to AprilTags.

Solution: go to config/baseline/pi_camera and change the framerate to 30 instead of 15.

  • [Warning] Topics ‘/![robot name]/camera_node/image/rect’ and ‘/![robot name]/camera_node/raw_camera_info’ do not appear to be synchronized. Solution:
Because of mathjax bug

Parallel Autonomy

✎🔗

???

Because of mathjax bug

Preliminaries

✎🔗

to write

Because of mathjax bug

Chapter template

✎🔗

Theory chapters benefit from a standardized exposition. Here, we define the template for these chapters. Rememeber to check Unit B-2 - Basic Markduck guide for a comprehensive and up-to-date list of Duckiebook supported features.

Example Title: PID control

✎🔗

Start with a brief introduction of the discussed topic, describing its place in the bigger picture, justifying the reading constraints/guidelines below. Write it as if the reader knew the relevant terminology. For example:

PID control is the simplest approach to making a system behave in a desired way rather than how it would naturally behave. It is simple because the measured output is directly feedbacked, as opposed to, e.g., the system’s states. The control signal is obtained as a weighted sum of the tracking error (_P_roportional term), its integral over time (_I_ntegrative term) and its instantaneous derivative (_D_erivative term), from which the appellative of PID control. The tracking error is defined as the instantaneous difference between a reference and a measured system output.

Knowledge necessary:

Required Reading: Insert here a list of topics and suggested resources related to necessary knowledge in order to understand the content presented. Example:

Terminology: autonomy overview

System Modeling: basic kinematics, basic dynamics, linear algebra, State space representations, Linear Time Invariant Systems

Suggested Reading: Insert here a list of topics and suggested resources related to recommended knowledge in order to better understand the content presented. Example:

Definitions of Stability, Performances and Robustness: [7], …

observability/detectability and controllability/reachability: [7]

Discrete time PID: [7]

Bode diagrams: [7]

Nyquist plots: [7]

[…]

Problem Definition

✎🔗

In this section we crisply define the problem object of this chapter. It serves as a very brief recap of exactly what is needed from previous atoms as well. E.g.

Let:

\begin{align} \dot{\state}_t = A\state_t+Bu_t \\ y = C\state_t+Du_t \label{eq:system}\tag{1} \end{align}

be the LTI model of the Duckiebot’s plant, with $x \in \statesp$, $y \in \mathbb{R}^p$ and $u \in \mathbb{R}^m$. We recall (Duckiebot Modeling) that:

\begin{align} A &= \left[ \begin{array}{ccc} a_{11} & \dots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{n1} & \dots & a_{nn} \end{array} \right] \\ B &= \left[ b_1 \,\, \dots \,\, b_m \right]^T \\ C &= \left[ c_1 \ \,\, \dots \,\, c_p \right] \\ D &= 0. \end{align}

[…]

Remember you can use the problem environment of $\LaTeX$ to formally state a problem:

PID Given a system \eqref{eq:system} and measurements of the output $\tilde{y}_t = y_t + n_t, n_t \sim \cal{N}(0,\sigma)$, find a set of PID coefficients that meet the specified requirements for: - stability, - performance, - robustness.

as shown in (Figure 1.1).

A classical block diagram for PID control. We like to use a lot of clear figures in the Duckiebook.

Introduced Notions

✎🔗

Examples

✎🔗

This section serves as a collection of theoretical and practical examples that can clarify part or all of the above.

Theoretical Examples

✎🔗

More academic examples

✎

Immagine a spring-mass-damper system…

✎

[…]

Implementation Examples

✎🔗

More Duckiebot related examples

✎ ✎

[…]

Pointers to Exercises

✎🔗

Here we just add references to the suggested exercises, defined in the appropriate exercise chapters.

References

✎🔗

Do not include a reference chapter. References are automatically compiled to the Bibliography Section.

Jacopo

Jacopo

Jacopo

Because of mathjax bug

Symbols and conventions

✎🔗

Andrea

Sets

✎🔗

Dzenan

k:sets

Maps

✎🔗

k:sets

k:maps

Numbers

✎🔗

k:sets

k:naturals, k:integers, k:reals

Natural numbers

✎🔗

$\nats = \{0, 1, 2, \cdots\}$

The natural numbers are the set positive numbers, including zero.

Given two natural their addition is always a natural number:

$$ a+b = c \in \nats, \forall a,b \in \nats. \label{eq:intro-nats}\tag{1}$$

The same does not hold of the subtraction operation:

$$ a-b = c \in \nats \iff a \geq b. $$

For this reason set of integer numbers is defined.

Complex numbers

✎🔗

Dzenan

Linearity and Vectors

✎🔗

Jacopo

Linear algebra provides the set of mathematical tools to (a) study linear relationships and (b) describe linear spaces. It is a field of mathematics with important ramifications.

Linearity is an important concept because it is powerful in describing the input-output behavior of many natural phenomena (or systems). As a matter of fact, all those systems that cannot be modeled as linear, still can be approximated as linear to gain an intuition, and sometimes much more, of what is going on.

So, in a way or the other, linear algebra is a starting point for investigating the world around us, and Duckietown is no exception.

This chapter is not intended to be a comprehensive compendium of linear algebra.

this reference

this other reference

add references throughout all chapter

Real numbers are complex for you?: Number theory addref

$\forall$ is a typo for A and $\in$ are Euros? Mathematical symbolic language.

Linearity

✎🔗

In this section we discuss vectors, matrices and linear spaces along with their properties.

Before introducing the these arguments, we need to formally define what we mean by linearity. The word linear comes from the latin linearis, which means pertaining to or resembling a line. You should recall that a line can be represented by an equation like $y = mx + q$, but here we intend linearity as a property of maps, so there is a little more to linearity than lines (although lines are linear maps indeed).

To avoid confusions, let us translate the concept of linearity in mathematical language.

Linearity A function $f: \aset{X} \to \aset{Y}$ is linear when, $\forall x_i \in \aset{X}$, $i = \{1,2\}$, and $\forall a \in \reals$:

\begin{align} f(ax_1) &= af(x_1), \label{eq:lin1}\tag{2} \quad \text{and:} \\ f(x_1 + x_2) &= f(x_1) + f(x_2) \label{eq:lin2}\tag{3} \end{align}

Condition \eqref{eq:lin1} is referred to as the property of homogeneity (of order 1), while condition \eqref{eq:lin2} is referred to as additivity.

Superposition Principle Conditions \eqref{eq:lin1} and \eqref{eq:lin2} can be merged to express the same meaning through: \begin{align} f(ax_1 + bx_2) = af(x_1) + bf(x_2), \forall x_i \in \aset{X}, i = \{1,2\}, \forall a,b \in \reals \label{eq:linearity}\tag{4}. \end{align}

This equivalent condition \eqref{eq:linearity} is instead referred to as superposition principle, which unveils the bottom line of the concept of linearity: adding up (equivalently, scaling up) inputs results in an added up (equivalently, scaled up) output.

Vectors

✎🔗

Let $n$ belong to the set of natural numbers $\nats$, i.e., $n \in \nats$, and let $a_i \in \reals$, $i = \{1, \dots, n\}$ be real coefficients. While $\reals$ is the set of real numbers, $\reals^n$ is the set of all $n$-tuples of real numbers.

Vector and components

An $n$-dimensional $\textit{vector}$ is an $n$-tuple:

\begin{align} \label{eq:vector}\tag{5} \avec{v} = \left[ \begin{array}{c} v_1 \\ \vdots \\ v_n \end{array} \right] \in \reals^{n \times 1} \equiv \reals^n, \end{align}

of components $v_1, \dots, v_n \in \reals$.

Vector notation A more general notation for tuples can be used when denoting vectors: \begin{align} \label{eq:vector-tuple-notation}\tag{6} \avec{v} = \tup{v_1, \cdots, v_n}. \end{align} In these preliminaries, we will adopt the \eqref{eq:vector} “engineering” notation as it arguably simplifies remembering vector-matrix operations (Unit G-9 - Matrices and vectors).

You can imagine a vector Figure 6.1 as a “directional number”, or an arrow that starts a certain point and goes in a certain direction (in $\reals^n$). In this representation, the number is the length of the arrow, or the magnitude of the vector (sometimes referred to even as modulus), and it can be derived through the vector’s components.

Length of a vector We define the length, or modulus, of a vector $\avec{v} \in \reals^n$ as: \begin{align} \label{eq:vec-2-norm}\tag{7} \|\avec{v}\| = \sqrt{v_1^2 + \dots + v_n^2} \in \reals. \end{align}

2-norm Generally speaking, it is not always possible to define the length of a vector (addref). But when it is possible (e.g., in Hilbert spaces), and in Duckietown it always is, there are many ways to define it. The most common and intuitive definition is the Euclidian- or 2-norm, which is defined above in \eqref{eq:vec-2-norm}.

We will discuss norms more in detail in Unit G-13 - Norms.

Unit vector A unit vector, or versor, is a vector $\avec{e}$ of of unit length: \begin{align} \label{eq:unit-vector}\tag{8} \|\avec{e}\| = 1. \end{align}

Unit vectors are used to define the directions of the components of a vector, allowing for an algebraic rather than vectorial representation. As we will see in Subsection 6.2.1 - Vector algebra, this will make the algebra of vectors more intuitive.

A vector, its components expressed as multiples of unit vectors.

Let $\avec{v} \in \reals^3$ be a vector defined in the Cartesian space. Let, moreover, $(\avec{i},\avec{j},\avec{k})^T$ be the versor of the Cartesian axis, i.e.: \begin{align}\label{eq:example-vector-algebraic}\tag{9} \avec{i} &= [1,0,0]^T; \\ \avec{j} &= [0,1,0]^T; \\ \avec{k} &= [0,0,1]^T. \end{align} Then, a vector can be written equivalently in vector or algebraic form: $\avec{v} = [v_1, v_2, v_3]^T = v_1\avec{i} + v_2\avec{j}+v_3\avec{k}$. Unit vectors are sometimes explicitly denoted with a hat (^), e.g., $\hat{\avec{i}}, \hat{\avec{j}}, \hat{\avec{k}}$.

Normalizing vectors Every vector can be made into a unit vector, or normalized, by dividing each of its components by the vector’s magnitude: \begin{align}\label{eq:vector-normalizing}\tag{10} \hat{\avec{v}} = \frac{\avec{v}}{\|\avec{v}\|} = \left[\frac{v_1}{\|\avec{v}\|}, \frac{v_2}{\|\avec{v}\|}, \frac{v_3}{\|\avec{v}\|}\right]^T. \end{align}

Vector algebra

✎🔗

We here define operations amongst two given vectors defined in the same space: $\avec{u} = [u_1, u_2, u_3]^T, \avec{v} = [v_1, v_2, v_3]^T \in \reals^3$.

Vectorial Sum

✎🔗

The sum of two vectors is a vector, and its components are the sum of the two vectors components.

Vectorial sum \begin{align} \label{eq:vector-sum}\tag{11} \avec{u} + \avec{v} = [u_1+v_1, u_2+v_2, u_3+v_3]^T. \end{align}

Sum Mathematical operations come in pairs, which represent the same concept. A sum operation, sometimes more extensively referred to as the algebric sum, is the concept of summing, i.e., it includes both addition and subtraction. (A subtraction is nothing but an addition between positive and negative numbers.)

The parallelogram law helps visualize the results of the vectorial sum operation Figure 6.2.

The sum of two vectors can be visualized with the parallelogram law.

Dot, or scalar, product

✎🔗

The dot, or scalar, product of two vectors ($\avec{u}$,$\avec{v} \in \mathbb{R}^3$) is a scalar ($a \in \mathbb{R}$) equal to the sum of the products of the components of the vectors. Equivalently, it can be expressed as the product of the magnitudes of the two vectors times the cosine of the angle between them, $\phi \in [0,2\pi)$.

Scalar product \begin{align} \label{eq:vector-dot-product}\tag{12} \avec{u} \cdot \avec{v} = u_1v_1+u_2v_2+u_3v_3 = \|u\|\|v\|\cos(\phi) \in \mathbb{R} \end{align}

The dot product is a measure of the projection of vectors on one another (Figure 6.3).

When the two vectors are perpendicular, or orthogonal, the dot product is zero ($\cos(\pi/2) = 0$). This fact is often used as a test for orthogonality. Orthogonality is an important concept for linear spaces, as the most “efficient” basis are orthogonal.

The scalar product between two vectors measures the projection of one on each other.

Cross, or vector, product

✎🔗

While the dot product depends on the metric chosen in the space (the Euclidian norm, in our case), the cross product even requires the definition of an orientation, or handedness.

Standard Basis In the Euclidian space $\mathbb{R}^3$, $\hat{\avec{i}}, \hat{\avec{j}}, \hat{\avec{k}}$ are the unit vectors for the standard basis, which is right handed.

In a right handed reference system such as the standard basis, the right hand rule (Figure 6.4) is the handy-est way to identify the direction of the vector resulting from a cross product.

ProTip: There is a valid reason for which it is called the right hand rule. Don’t use your left hand because you are holding a pen with the right one.

The right hand rule points in the direction of the resulting vector from a cross product.

The cross, or vector, product between two vectors ($\avec{u}$, $\avec{v} \in \mathbb{R}^3$) is a vector that is orthogonal to each of the two vectors, hence is normal, or perpendicular, to the plane containing them. Its magnitude is given by the product of their magnitude times the sine of the angle between them, and its direction is indicated by the normal unit vector ($\hat{\avec{n}} \in \mathbb{R}^3$), identified by the right hand rule.

Vector product \begin{align} \label{eq:vector-cross-product}\tag{13} \avec{u} \times \avec{v} = [u_2v_3-u_3v_2, u_3v_1-u_1v_3, u_1v_2-u_2v_1]^T = \|u\|\|v\|\sin(\phi) \hat{\avec{n}}. \end{align}

Geometric interpretation A cross product encodes two pieces on information: a direction, which is orthogonal to the plane spanned by the two vectors, and a magnitude, which is equal to the area of the parallelogram having $\avec{u}$, and $\avec{v}$ as sides.

Keeping \eqref{eq:vector-cross-product} and Remark 7 - Geometric interpretation in mind, it should be intuitive to understand that: \begin{align} \label{eq:vec-cross-vv-v0}\tag{14} \avec{v} \times \avec{v} &= \avec{0}, \forall \avec{v} \in \mathbb{R}^n, \\ \avec{v} \times \avec{0} &= \avec{0}, \forall \avec{v} \in \mathbb{R}^n. \end{align}

The zero vector ($\avec{0}$) is a vector with zero magnitude, not the same as the number zero ($0$).

Each component of $\avec{w}$ is the difference of the products of the two other components of $\avec{u}$, and $\avec{v}$, in the order given by the chosen handedness of the basis. This combination resembles a cross (Figure 6.5), from which the name of cross product.

Each component of the resulting vector is the product of the alternated other components, forming a cross.

The components of a cross product can be computed through the Sarrus rule (see Section 7.6 - Determinant).

As consequence of the vectorial product’s definition and right handedness of the basis, the following hold true in the Cartesian space:

\begin{align} \label{eq:sb-cross-products}\tag{15} \hat{\avec{i}} \times \hat{\avec{j}} &= \hat{\avec{k}} \\ \hat{\avec{j}} \times \hat{\avec{k}} &= \hat{\avec{i}} \\ \hat{\avec{k}} \times \hat{\avec{i}} &= \hat{\avec{j}}. \end{align}

Properties of vectors

✎🔗

In this section we highlight the properties of vector operations, that derive from their definitions.

✎

The vector sum obejs the following:

  • $\avec{u} + \avec{v} = \avec{v} + \avec{u}$,
  • $(\avec{u} + \avec{v}) + \avec{w} = \avec{u} + (\avec{v} + \avec{w})$,
  • $a(\avec{u} + \avec{v}) = a\avec{u} + a\avec{v}$,
  • $(a+b)\avec{u} = a\avec{u} + b\avec{u}$,
  • $\avec{u} + \avec{0} = \avec{u}$, therefore $\avec{u} + (-\avec{u}) = \avec{0}$.
✎

Letting $\phi \in [0,2\pi)$ be the angle between two vectors $\avec{u}, \avec{v}$, the dot product obejs the following:

  • $\avec{u} \cdot \avec{v} = \| \avec{u} \|\| \avec{v} \|\cos(\phi)$,
  • $\avec{u} \cdot \avec{u} = \| \avec{u} \|^2$,
  • $\avec{u} \cdot \avec{v} = \avec{v} \cdot \avec{u}$,
  • $\avec{u} \cdot (\avec{v} + \avec{w}) = \avec{u} \cdot \avec{v} + \avec{u} \cdot \avec{w}$,
  • $a (\avec{u} \cdot \avec{v}) = (a\avec{u}) \cdot \avec{v}$,
  • $\avec{0} \cdot \avec{u} = 0$
  • $\avec{u} \cdot \avec{v}$ = 0 $\iff$ $\avec{u}=\avec{0}$, $\avec{v}=\avec{0}$, or $\avec{u} \bot \avec{v}$.
✎

Letting $\phi \in [0,2\pi)$ be the angle between two vectors $\avec{u}, \avec{v}$, the cross product obejs the following:

  • $\avec{u} \times \avec{v} = \| \avec{u} \|\| \avec{v} \|\sin(\phi) \hat{\avec{n}}$,
  • $\avec{u} \times \avec{v} = - \avec{v} \times \avec{u}$,
  • $(a\avec{u}) \times \avec{v} = \avec{u} \times (a\avec{v}) = a(\avec{u} \times \avec{v})$,
  • $\avec{u} \times (\avec{v} + \avec{w}) = \avec{u} \times \avec{v} + \avec{u} \times \avec{w}$,
  • $\avec{u} \cdot (\avec{v} \times \avec{w}) = (\avec{u} \times \avec{v}) \cdot \avec{w}$,
  • $\avec{u} \times (\avec{v} + \avec{w}) = (\avec{w} \cdot \avec{u}) \avec{v} - (\avec{v} \cdot \avec{u}) \avec{w} \neq (\avec{u} \times \avec{v}) + \avec{w}$,
  • $\avec{u} \times \avec{v} = 0 \iff \avec{u}=\avec{0}, \avec{v}=\avec{0}$, or $\avec{u} \parallel \avec{v}$.

Linear dependance

✎🔗

Linear dependance Two or more vectors $\{\avec{v_1},\cdots,\avec{v_n}\}$ are linearly dependant if there exists a set of scalars $\{a_1, \cdots, a_k\}, k \leq n$, that are not all zero, such that: $$ \label{eq:lin-dep}\tag{1} a_1\avec{v_1} + \cdots + a_k\avec{v_k} = \avec{0}. $$

When \eqref{eq:lin-dep} is true, it is possible to write at least one vector as a linear combination of the others.

Linear independance Two or more vectors ${\avec{v_1},\cdots,\avec{v_n}}$ are linearly independant if \eqref{eq:lin-dep} can be satisfied only by $k=n$ and $a_i =0, \forall i = 1, \cdots, n$.

Pointers to Exercises

✎🔗

Here we just add references to the suggested exercises, defined in the appropriate exercise chapters.

add exercises

Matrices basics

✎🔗

Dzenan

k:basic_math

k:linear_algebra

k:matrices

A matrix:

$$ \amat{A} = \left[ \begin{array}{ccc} a_{11} & \dots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{m1} & \dots & a_{mn} \end{array} \right] \in \reals^{m \times n} \label{eq:matrix}\tag{1} $$

is a table ordered by ($m$) horizontal rows and ($n$) vertical columns. Its elements are typically denoted with lower case latin letters, with subscripts indicating their row and column respectively. For example, $a_{ij}$ is the element of $A$ at the $i$-th row and $j$-th column.

A matrix. This image is taken from [17]

A vector is a matrix with one column.

Matrix dimensions

✎🔗

The number of rows and columns of a matrix are referred to as the matrix dimensions. $\amat{A} \in \reals^{m \times n}$ has dimensions $m$ and $n$.

Fat matrix When $n \gt{} m$, i.e., the matrix has more columns than rows, $\amat{A}$ is called fat matrix.

Tall matrix When $n \lt{} m$, i.e., the matrix has more rows than columns, $\amat{A}$ is called tall matrix.

Fat matrix When $n = m$, $\amat{A}$ is called square matrix.

Square matrices are particularly important.

Diagonal matrix

✎🔗

Diagonal matrix A diagonal matrix has non zero elements only on its main diagonal. \begin{align} \amat{A} = \left[ \begin{array}{ccc} a_11 & 0 & \dots & 0 \\ 0 & a_22 & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \dots & 0 & a_nn \end{array} \right] \end{align}

Identity matrix

✎🔗

Identity matrix An identity matrix is a diagonal square matrix with all elements equal to one. \begin{align} \amat{I} = \left[ \begin{array}{ccc} 1 & 0 & \dots & 0 \\ 0 & 1 & \ddots & \vdots \\ \vdots & \ddots & \ddots & 0 \\ 0 & \dots & 0 & 1 \end{array} \right] \end{align}

Null matrix

✎🔗

Null matrix The null, or Zero, matrix is a matrix whos elements are all zeros. \begin{align} \amat{0} = \left[ \begin{array}{ccc} 0 & \dots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \dots & 0 \end{array} \right] \end{align}

Determinant

✎🔗
  • 2x2
  • 3x3
  • nxn

Rank of a matrix

✎🔗

Trace of a matrix

✎🔗
Because of mathjax bug

Matrix inversions

✎🔗

Dzenan

Matrix Inverse

✎🔗

Square matrix:

$\amat{A}\amat{A}^{-1} = \amat{I}$

Matrices and vectors

✎🔗

Dzenan

  • matrix-vector product

Matrix as representation of linear (vector) spaces

✎🔗
  • linear system to matrix representation
  • linearly dependent and independent spaces

Fundamental spaces

✎🔗
  • Null space
  • Range/image

Eigenvalues and Eigenvectors

✎🔗
  • for square matrices

  • for rectangular matrices (topic for advanced-linear-algebra?)

  • condition number of a matrix (?)

Because of mathjax bug

Matrix operations (basic)

✎🔗

Dzenan

  • sum of matrices
  • product of matrices
  • matrix transpose – Symmetric matrix {#mat-sym}
  • matrix concatenation

Properties

✎🔗
Because of mathjax bug

Matrix operations (complex)

✎🔗

Dzenan

  • matrix scalar product
  • matrix Hadamart product
  • matrix power
  • matrix exponential

Properties

✎🔗
Because of mathjax bug

Matrix diagonalization

✎🔗

Dzenan

  • singular value decomposition SVD (topic for advanced-linear-algebra?)
✎
  • show how to diagonalize matrices and why it is relevant (it will come in handy for state space representation chapter chapter)

Norms

✎🔗

Dzenan

finish writing

Other metrics can be defined to measure the “length” of a vector. Here, we report some commonly used norms. For a more in depth discussion of what constitutes a norm, and their properties, see:

[18].

$p$-norm

✎🔗

Let $p \geq 1 \in \reals$. The $p$-norm is defined as:

$p$-norm \begin{align} \label{eq:vec-p-norm}\tag{1} \|\avec{v}\|_p = \displaystyle \left( \sum_{i=1}^{n} |v_i|^p \right)^{\frac{1}{p}}. \end{align}

The $p$-norm is a generalization of the $2$-norm ($p=2$ in \eqref{eq:p-norm}) introduced above (Definition 19 - Length of a vector). The following $1$-norm and $\infty$-norm can as well be obtained from \eqref{eq:vec-p-norm} with $p=1$ and $p \rightarrow \infty$ respectively.

One norm

✎🔗

The $1$-norm is the sum of the absolute values of a vector’s components. It is sometimes referred to as the Taxicab norm, or Manhattan distance as it well describes the distance a cab has to travel to get from a zero starting point to a final destination $v_i$ on a grid.

$1$-norm Given a vector $\avec{v} \in \reals^n$, the $1$-norm is defined as: \begin{align} \label{eq:vec-one-norm}\tag{2} \|\avec{v}\| = \displaystyle \sum_{i=1}^{n}|v_i|. \end{align}

$\infty$-norm

✎🔗

The infinity norm measures the maximum component, in absolute value, of a vector.

$\infty$-norm \begin{align} \label{eq:vec-inf-norm}\tag{3} \|\avec{v}\| = \displaystyle \max(|v_1|, \cdots, |v_n|). \end{align}

Definition

✎🔗

Properties

✎🔗
Because of mathjax bug

From ODEs to LTI systems

✎🔗

Miguel

Linearization

✎🔗

Miguel

Probability basics

✎🔗

In this chapter we give a brief review of some basic probabilistic concepts. For a more in-depth treatment of the subject we refer the interested reader to a textbook such as [19].

Random Variables

✎🔗

The key underlying concept in probabilistic theory is that of an event, which is the output of a random trial. Examples of an event include the result of a coin flip turning up HEADS or the result of rolling a die turning up the number “4”.

Random Variable A (either discrete or continuous) variable that can take on any value that corresponds to the feasible output of a random trial.

For example, we could model the event of flipping a fair coin with the random variable $X$. We write the probability that $X$ takes HEADS as $p(X=\text{HEADS})$. The set of all possible values for the variable $X$ is its domain, $\aset{X}$. In this case, $$ \aset{X}=\{\text{HEADS},\text{TAILS}\}. $$ Since $X$ can only take one of two values, it is a binary random variable. In the case of a die roll, $$ \aset{X}=\{1,2,3,4,5,6\}, $$ and we refer to this as a discrete random variable. If the output is real value or a subset of the real numbers, e.g., $\aset{X} = \reals$, then we refer to $X$ as a continuous random variable.

Consider once again the coin tossing event. If the coin is fair, the have $p(X=HEADS)=p(X=TAILS)=0.5$. Here, the function $p(x)$ is called the probability mass function or pmf. The pmf is shown in Figure 16.1.

The pmf for a fair coin toss

Here are some very important properties of $p(x)$: - $0\leq p(x) \leq (1)$ - $\sum_{x\in\aset{X}}=1$

In the case of a continuous random variable, we will call this function $f(x)$ and call it a probability density function, or pdf.

In the case of continuous RVs, technically the $p(X=x)$ for any value $x$ is zero since $\aset{X}$ is infinite. To deal with this, we also define another important function, the cumulative density function, which is given by $F(x) \triangleq p(X\leq x)$, and now we can define $f(x) \triangleq \frac{d}{dx}F(x)$. A pdf and corresponding cdf are shown in Figure 16.2 (This happens to be a Gaussian distribution, defined more precisely in Subsection 16.1.8 - The Gaussian Distribution.).

The continuous pdf and cdf

Joint Probabilities

✎🔗

If we have two different RVs representing two different events $X$ and $Y$, then we represent the probability of two distinct events $x \in \aset{X}$ and $y \in \mathcal{Y}$ both happening, which we will denote as following: $p(X=x \; \text{AND} \; Y=y) = p(x,y)$. The function $p(x,y)$ is called joint distribution.

Conditional Probabilities

✎🔗

Again, considering that we have to RVs, $X$ and $Y$, imagine these two events are linked in some way. For example, $X$ is the numerical output of a die roll and $Y$ is the binary even-odd output of the same die roll. Clearly these two events are linked since they are both uniquely determined by the same underlying event (the rolling of the die). In this case, we say that the RVs are dependent on one another. In the event that we know one of events, this gives us some information about the other. We denote this using the following conditional distribution $p(X=x \; \text{GIVEN} \; Y=y) \triangleq p(x|y)$.

Write down the conditional pmf for the scenario just described assuming an oracle tells you that the die roll is even. In other words, what is p(x|\text{EVEN})?

(Warning: if you think this is very easy that’s good, but don’t get over-confident.)

The joint and conditional distributions are related by the following (which could be considered a definition of the joint distribution):

\begin{equation} p(x,y) = p(x|y)p(y) \label{eq:joint}\tag{1} \end{equation}

and similarly, the following could be considered a definition of the conditional distribution:

\begin{equation} p(x|y) = \frac{p(x,y)}{p(y)} \; \text{if} \; p(y) \gt{} 0 \label{eq:condition}\tag{2} \end{equation}

In other words, the conditional and joint distributions are inextricably linked (you can’t really talk about one without the other).

If two variables are independent, then the following relation holds: $p(x,y)=p(x)p(y)$.

Bayes’ Rule

✎🔗

Upon closer inspection of \eqref{eq:joint}, we can see that the choice of which variable to condition upon is completely arbitrary. We can write:

$$ p(y|x)p(x) = p(x,y) = p(x|y)p(y) $$

and then after rearranging things we arrive at one of the most important formulas for mobile robotics, Bayes’ rule:

\begin{equation} p(x|y) = \frac{p(y|x)p(x)}{p(y)} \label{eq:bayes}\tag{3} \end{equation}

Exactly why this formula is so important will be covered in more detail in later sections (TODO), but we will give an initial intuition here.

Consider that the variable $X$ represents something that we are trying to estimate but cannot observe directly, and that the variable $Y$ represents a physical measurement that relates to $X$. We want to estimate the distribution over $X$ given the measurement $Y$, $p(x|y)$, which is called the posterior distribution. Bayes’ rule lets us to do this. For every possible state, you take the probability that this measurement could have been generated, $p(y|x)$, which is called the measurement likelihood, you multiply it by the probability of that state being the true state, $p(x)$, which is called the prior, and you normalize over the probability of obtaining that measurement from any state, $p(y)$, which is called the evidence.

From Wikipedia: Suppose a drug test has a 99% true positive rate and a 99% true negative rate, and that we know that exactly 0.5% of people are using the drug. Given that a person’s test gives a positive result, what is the probability that this person is actually a user of the drug.

Answer: $\approx$ 33.2%. This answer should surprise you. It highlights the power of the prior.

Conditional Independence

✎🔗

Two RVs, $X$ and $Y$ may be correlated, we may be able to encapsulate the dependence through a third random variable $Z$. Therefore, if we know $Z$

A graphical representation of the conditional independence of $X$ and $Y$ given $Z$
comment

Is there a discussion of graphical models anywhere? Doing a good job of sufficiently describing graphical models and the dependency relations that they express requires careful thought. Without it, we should refer readers to a graphical models text (e.g., Koller and Friedman, even if it is dense)

Moments

✎🔗

The $n$th moment of an RV, $X$, is given by $E[X^n]$ where $E[]$ is the expection operator with:

$$ E[f(X)] = \sum_{\aset{X}} x \, f(x) $$ in the discrete case and $$ E[f(X)] = \int x\, f(x) dx $$ in the continuous case.

The 1st moment is the mean, $\mu_X=E[X]$.

The $n$th central moment of an RV, $X$ is given by $E[(X-\mu_X)^n]$. The second central moment is called the covariance, $\sigma^2_X=E[(X-\mu_X)^2]$.

Entropy

✎🔗

The entropy of an RV is a scalar measure of the uncertainty about the value the RV.

A common measure of entropy is the Shannon entropy, whose value is given by

\begin{equation} H(X)=-E[\log_2 p(x)] \label{eq:shannon}\tag{4} \end{equation}

This measure originates from communication theory and literally represents how many bits are required to transmit a distribution through a communication channel. For many more details related to information theory we recommend [20].

As an example, we can easily write out the Shannon entropy associated with a binary RV (e.g. flipping a coin) as a function of the probability that the coin turns up heads (call this $p$):

\begin{equation} H(X) = -p\log_2 p - (1-p)\log_2 (1-p) \label{eq:binary_entropy}\tag{5} \end{equation}

The Shannon entropy of a binary RV $X$

Notice that our highest entropy (uncertainty) about the outcome of the coin flip is when it is a fair coin (equal probability of heads and tails). The entropy decays to 0 as we approach $p=0$ and $p=1$ since in these two cases we have no uncertainty about the outcome of the flip. It should also be clear why the function is symmetrical around the $p=0.5$ value.

The Gaussian Distribution

✎🔗

In mobile robotics we use the Gaussian, or normal, distribution a lot.

comment

The banana distribution is the official distribution in robotics! - AC

comment

The banana distribution is Gaussian! http://www.roboticsproceedings.org/rss08/p34.pdf - LP

The 1-D Gaussian distribution pdf is given by:

\begin{equation} \mathcal{N}(x|\mu,\sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2} \label{eq:gaussian1D}\tag{6} \end{equation}

where $\mu$ is called the mean of the distribution, and $\sigma$ is called the standard deviation. A plot of the 1D Gaussian was previously shown in Figure 16.2.

We will rarely deal with the univariate case and much more often deal with the multi-variate Gaussian:

\begin{equation} \mathcal{N}(\state|\bmu,\bSigma) = \frac{1}{(2*\pi)^{D/2}|\bSigma|^{1/2}}\exp[-\frac{1}{2}(\state-\bmu)^T\bSigma^{-1}(\state - \bmu)] \label{eq:gaussianND}\tag{7} \end{equation}

The value from the exponent: $(\state-\bmu)^T\bSigma^{-1}(\state - \bmu)$ is sometimes written $||\state - \bmu||_\bSigma$ and is referred to as the Mahalanobis distance or energy norm.

Mathematically, the Gaussian distribution has some nice properties as we will see. But is this the only reason to use this as a distribution. In other words, is the assumption of Gaussianicity a good one?

There are two very good reasons to think that the Gaussian distribution is the “right” one to use in a given situation.

  1. The central limit theorem says that, in the limit, if we sum an increasing number of independent random variables, the distribution approaches Gaussian

  2. It can be proven (TODO:ref) that the Gaussian distribution has the maximum entropy subject to a given value for the first and second moments. In other words, for a given mean and variance, it makes the least assumptions about the other moments.

Exercise: derive the formula for Gaussian entropy

Because of mathjax bug

Kinematics

✎🔗

Harshit

Kinematics is the study of position, velocity and acceleration of geometric points.

A point has no dimensions. For example, the center of mass of a body is a point.

Because of mathjax bug

Dynamics

✎🔗

Harshit

Because of mathjax bug

Coordinate systems

✎🔗

Reference frames

✎🔗

Required Reading: The following assumes a working familiarity with 2D and 3D Cartesian coordinate systems. If you are not familiar with Cartesian coordinate systems, please read the chapter on coordinate systems.

Time series

✎🔗

Knowledge of k:tuple

Knowledge of time series.

Knowledge of time series operations, such as upsampling and downsampling.

Transformations

✎🔗

Required Reading: The following assumes a working familiarity with 2D and 3D Cartesian reference frames. If you are not familiar with Cartesian reference frames, please read the chapter on reference frames. Some familiarity with linear alegra is also helpful.

Types

✎🔗

Introduction to type theory.

Knowledge of set theory.

Computer science concepts

✎🔗

This unit will contain brief explanations / pointers to basic concepts of computer science.

Real-time Operating system

✎🔗

to write

Because of mathjax bug

A course in autonomy

✎🔗

These are the theory units.

Because of mathjax bug

Autonomous Vehicles

✎🔗

Liam

Autonomous Vehicles in the News

✎🔗

These days it is hard to separate the fact from the fiction when it comes to autonomous vehicles, particularly self-driving cars. Virtually every major car manufacturer has pledged to deploy some form of self-driving technology in the next five years. In addition, there are many startups and software companies which are also known to be developing self-driving car technology.

Here’s a non-exhaustive list of some of companies that are actively developing autonomous cars:

Levels of Autonomy

✎🔗

Before even discussing any detailed notion of autonomy, we have to specify exactly what we are talking about. In the United States, the governing body is the NHTSA, and they have recently (Oct 2016) redefined the so-called “levels of autonomy” for self-driving vehicles.

In broad terms, they are as follows

  • Level 0: the human driver does everything;
  • Level 1: an automated system on the vehicle can sometimes assist the human driver conduct some parts of the driving task;
  • Level 2: an automated system on the vehicle can actually conduct some parts of the driving task, while the human continues to monitor the driving environment and performs the rest of the driving task;
  • Level 3: an automated system can both actually conduct some parts of the driving task and monitor the driving environment in some instances, but the human driver must be ready to take back control when the automated system requests;
  • Level 4: an automated system can conduct the driving task and monitor the driving environment, and the human need not take back control, but the automated system can operate only in certain environments and under certain conditions; and
  • Level 5: the automated system can perform all driving tasks, under all conditions that a human driver could perform them.
Because of mathjax bug

Autonomy overview

✎🔗

Liam

This unit introduces some basic concepts ubiquitous in autonomous vehicle navigation.

Basic Building Blocks of Autonomy

✎🔗

The minimal basic backbone processing pipeline for autonomous vehicle navigation is shown in Figure 2.1.

The basic building blocks of any autonomous vehicle

For an autonomous vehicle to function, it must achieve some level of performance for all of these components. The level of performance required depends on the task and the required performance. In the remainder of this section, we will discuss some of the most basic options. In the next section we will briefly introduce some of the more advanced options that are used in state-of-the-art autonomous vehicles.

Sensors

✎🔗
Velodyne 3D Laser Scanner
Camera
Automotive Radar
GPS Receiver
Inertial Measurement Unit
Some common sensors used for autonomous navigation

Sensor A sensor is a device that or mechanism that is capable of generating a measurement of some external phyiscal quantity

In general, sensors have two major types. Passive sensors generate measurements without affecting the environment that they are measuring. Examples include inertial sensors, odometers, GPS receivers, and cameras. Active sensors emit some form of energy into the environment in order to make a measurement. Examples of this type of sensor include Light Detection And Ranging (LiDAR), Radio Detection And Ranging (RaDAR), and Sound Navigation and Ranging (SoNAR). All of these sensors emit energy (from different spectra) into the environment and then detect some property of the energy that is reflected from the environment (e.g., the time of flight or the phase shift of the signal)

Raw Data Processing

✎🔗

The raw data that is input from a sensor needs to be processed in order to become useful and even understandandable to a human.

First, calibration is usually required to convert units, for example from a voltage to a physical quantity. As a simple example consider a thermometer, which measures temperature via an expanding liquid (usually mercury). The calibration is the known mapping from amount of expansion of liquid to temperature. In this case it is a linear mapping and is used to put the markings on the thermometer that make it useful as a sensor.

We will distiguish between two fundamentally types of calibrations.

Intrinsic Calibration An intrinsic calibration is required to determine sensor-specific paramaters that are internal to a specific sensor.

Extrinsic Calibration An extrinsic calibration is required to determine the external configuration of the sensor with respect to some reference frame.

For more information about reference frames check out Unit G-20 - Reference frames

Calibration is very important consideration in robotics. In the field, the most advanced algorithms will fail if sensors are not properly calibrated.

Once we have properly calibrated data in some meaningful units, we often do some preprocessing to reduce the overall size of the data. This is true particularly for sensors that generate a lot of data, like cameras. Rather than deal with every pixel value generated by the camera, we will process an image to generate feature-points of interest. In “classical” computer vision many different feature descriptors have been proposed (Harris, BRIEF, BRISK, SURF, SIFT, etc), and more recently Convolutional Neural Networks (CNNs) are being used to learn these features.

The important property of these features is that they should be as easily to associate as possible across frames. In order to achieve this, the feature descriptors should be invariant to nuissance parameters.

Top: A raw image with feature points indicated. Bottom: Lines projected onto ground plane using extrinsic calibration and ground projections

State Estimation

✎🔗

Now that we have used our sensors to generate a set of meaningful measurements, we need to combine these measurements together to produce an estimate of the underlying hidden state of the robot and possibly to environment.

State The state $\state_t \in \statesp$ is a sufficient statistic of the environment, i.e. it contains all sufficient information required for the robot to carry out its task in that environment. This can (and usually does) include the configuration of the robot itself.

What variables are maintained in the statespace $\statesp$ depends on the problem at hand. For example we may just be interested in a single robot’s configuration in the plane, in which case $\state_t \equiv \pose_t$. However, in other cases, such as simultaneous localization and mapping, me may also be tracking the map in the state space.

According to Bayesian principles, any system parameters that are not fully known and deterministic should be maintained in the state space.

In general, we do not have direct access to values in $\state$, instead we rely on our (noisy) sensor measurements to tell us something about them, and then we infer the values.

The video is at https://vimeo.com/232324847.

Lane Following in Duckietown. *Top Right*: Raw image; *Bottom Right*: Line detections; *Top Left*: Line projections and estimate of robot position within the lane (green arrow); *Bottom Left*: Control signals sent to wheels.

The animation in Figure 2.4 shows the lane following procedure. The output of the state estimator produces the green arrow in the top left pane.

Actuation

✎🔗

The very innermost control loop deals with actually tracking the correct voltage to be sent to the motors. This is generally executed as close to the hardware level as possible. For example we have a Stepper Motor HAT See the parts list.

Infrastructure and Prior Information

✎🔗

In general, we can make the autonomous navigation a simpler one by exploiting existing structure, infastructure, and contextual prior knowledge.

Infrastructure example: Maps or GPS satellites

Structure example: Known color and location of lane markings

Contextual prior knowledge example: Cars tend to follow the Rules of the Road

Advanced Building Blocks of Autonomy

✎🔗

The basic building blocks enable static navigation in Duckietown. However, many other components are necessary for more realistic scenarios.

Object Detection

✎🔗
Advanced Autonomy: Object Detection

One key requirement is the ability to detect objects in the world such as but not limited to: signs, other robots, people, etc.

SLAM

✎🔗

The simultaneous localization and mapping (SLAM) problem involves simultaneously estimating not only the robot state but also the map at the same time, and is a fundamental capability for mobile robotics. In autonomous driving, generally the most common application for SLAM is actual in the map-building task. Once a map is built then it can be pre-loaded and then used for pure localization. A demonstration of this in Duckietown is shown in Figure 2.9.

The video is at https://vimeo.com/232333888.

Localization in Duckietown

Modern Robotic Systems

✎🔗

Andrea Censi

The many parts of a robotic system

Modern robotics development

Example of cloud learning, annotators in the loop scenario.

System architectures basics

✎🔗

Andrea Censi

Physical and logical architectures.

Deployment as mapping a physical architecture onto the logical.

Basic graph concepts.

Basic operating systems concepts.

Logical and physical architecture

✎🔗

When we design a robotic system, or any cyber-physical system, we distinguish between what we shall call “logical architecture” and the “physical architecture”.

The logical architecture describes how the functionality is divided in abstract modules, and what these modules communicate.

The physical architecture describes how the modules are instantiated. For example, this includes the information of how many processors are used, and which processor runs which routine.

Logical architecture

✎🔗

The logical architecture is independent of the implementation (the hardware, the language.)

The logical architecture describes:

  • The system decomposition in components
  • The data flow: who tells whom what
  • How knows what: other sources of information, such as priors.
  • How information is represented

A logical architecture would also describe what are the representations used. This is explored more fully in the unit about representations.

Physical architecture

✎🔗

Processors

Buses / networks

Middleware

Orchestrators

In ROS that is the roscore program.

Autonomy architectures

✎🔗

Andrea Censi

Representations

✎🔗

Matt

Required Reading: The following assumes working knowledge of 2D and 3D Cartesian coordinate systems, reference frames, and coordinate transformations. If you are not familiar with these topics, please see the following preliminary chapters.

Coordinate systems

Reference frames

Coordinate transformations.

Robots are embodied autonomous agents that interact with a physical environment.

As you read through this book, you will find that shared representations of the agent (i.e., the robot) and the environment in which it operates are fundamental to a robot’s ability to sense, plan, and act—the capabilities that are key to making a robot a robot.

The state $\state_t \in \statesp$ is a representation that consists of a compilation of all knowledge about the robot and its environment that is sufficient both to perform a particular task as well as to predict the future. Of course, “predict the future” is vague, and we can not expect the state to include knowledge to predict everything about the future, but rather what is relevant in the context of the task.

The state $\state_t \in \statesp$ is a representation of the robot and the environment that is sufficient to predict the future in the context of the task being performed.

comment

I understand that explicitly referencing the task as relevant to the notion of state is odd, but I want to convey that the representation that we choose for the state does depend upon the task.

Now, let’s be a bit more formal with regards to what we mean by a state being “sufficient” to predict the future. We are specifically referring to a state that exhibits the Markov property, i.e., that it provides a complete summary of the past (again, in the context of the task). Mathematically, we say that a state is Markov if the future state is independent of the past given the present state (technically, this corresponds to first-order Markov):

\begin{equation} p(x_{t+1} \vert x_t, a_t, x_{t-1}, a_{t-1}, \ldots, x_0, a_0) = p(x_{t+1} \vert x_t, a_t) \end{equation}

A state exhibits the Markov property if and only if the above holds.

comment

I usually say that “state” is all that is needed to predict the future. The agent only needs to keep track of a smaller thing than the state to act optimally. There isn’t a good name to use; but it should be distinct from “state”.

comment

I think that this oversimplifies things. What is it about the future that is being predicted? Certainly, the states used by autonomous systems aren’t sufficient to predict everything about the future (e.g., a self-driving car can’t predict whether a street light is going to come on, but that probably doesn’t matter).

Knowledge about the robot and its environment is often extracted from the robot’s multimodal sensor streams, such as wheel encoders and cameras. Consequently, one might choose to formulate the state as the collection of all of the measurements that the robot acquires over time. Indeed, the use of low-level sensor measurements as the representation of the state has a long history in robotics and artificial intelligence and has received renewed attention of-late, notably in the context of deep learning approaches to visuomotor policy learning.

However, while measurement history is a sufficient representation of the robot and its operating environment, it serves as a challenging definition of state.

First, raw measurements are redundant, both within a single observation and across successive measurements. For example, one doesn’t need to reason over all pixels in an image, let alone all pixels within successive images to understand the location of a street sign.

Second, raw measurements contain a large amount of information that is not necessary for a given task. For example, the pixel intensities that capture the complex diffusion light due to clouds convey information that is not useful for self-driving cars. Requiring algorithms that deal with state to reason over these pixels would be unnecessarily burdensome.

Third, measurement history is very inefficient: its size grows linearly with time as the robot makes new observations, and it may be computationally intractable to access and process such a large amount of data.

This motivates the desire for a minimal representation that expresses knowledge that is both necessary and sufficient for the robot to perform a given task. More concretely, we will consider parameterized (symbolic) formulations of the state and will prefer representations that involve as few parameters as possible, subject to the state being Markov and the constraints imposed by the task.

Metric spaces (namely, Euclidean spaces) constitute the most commonly adopted state space in robotics, be it through the use of feature-based parameterizations of the state or gridmap representation. However, it is not uncommon to define the state of the robot and its environment in terms of a topological space or a hybrid metric-topological space.

Importantly, the exteroceptive and proprioceptive sensor measurements from which the robot’s perception algorithms infer the state are noisy, the models that describe the motion of the robot and environment are error-prone, and many aspects of the state are not directly observable (e.g., your Duckiebot may not be able to “see” some of the other Duckiebots on the road due to occlusions or the cameras limited field-of-view). As a result, robots must reason over probabilistic models of the state, commonly referred to as the belief, in order to effectively mitigate this uncertainty.

Modern signal processing

✎🔗

Andrea Censi

Understanding of time series. See: Unit G-21 - Time series.

Understanding of the basic concepts of event-based processing and why it is different from periodic processing.

Understanding of latency, frequency.

Middleware

✎🔗

Andrea

Knowledge of middleware, marshalling, service discovery.

Modularization and Contracts

✎🔗

Andrea Censi

Configuration management

✎🔗

Andrea

How to deal with configuration effectively.

Duckiebot modeling

✎🔗

Obtaining a mathematical model of the Duckiebot is important in order to (a) understand its behavior and (b) design a controller to obtain desired behaviors and performances, robustly.

The Duckiebot uses an actuator (DC motor) for each wheel. By applying different torques to the wheels a Duckiebot can turn, go straight (same torque to both wheels) or stay still (no torque to both wheels). This driving configuration is referred to as differential drive.

In this section we will derive the model of a differential drive wheeled robot. The Duckiebot model will receive voltages as input (to the DC motors) and produce a configuration, or pose, as output. The pose describes unequivocally the position and orientation of the Duckiebot with respect to some Newtonian “world” frame.

Different methods can be followed to obtain the Duckiebot model, namely the Lagrangian or Newton-Euler, we choose to describe the latter as it arguably provides a clearer physical insight. Showing the equivalence of these formulations is an interesting exercises that the interested reader can take as a challenge. A useful resource for modeling of a Duckiebot may be found here [27].

Requires:k:reference_frames (inertial, body), k:intro-transformations (Cartesian, polar)

k:intro-kinematics

k:intro-dynamics

Suggested: k:intro-ODEs-to-LTIsys

k:diff-drive-robot-model

Preliminaries

✎🔗

relabel inertial frame -> local frame; $(\cdot)^I \rightarrow (\cdot)^L$

We first briefly recapitulate on the (reference frames)[#reference-frames] that we will use to model the Duckiebot, with the intent of introducing the notation used throughout this chapter. It is important to note that we restrict the current analysis to the plane, so all of the following in defined in $\reals^2$.

To describe the behavior of a Duckiebot three reference frames will be used:

  • A “world” frame: a right handed fixed reference system with origin in some arbitrary point $O$. We will indicate variables expressed in this frame with a superscript $W$, e.g., $X^W$, unless there is no risk of ambiguity, in which case no superscript will be used.

  • An “inertial” frame: a fixed reference system parallel to the “world” frame, that spans the plane on which the Duckiebot moves. We will denote its axis as $\{X_I, Y_I\}$, and it will have origin in point $A=(x_A, y_A)$, i.e., the midpoint of the robot’s wheel axle. We will indicate variables expressed in this frame with a superscript $I$, e.g., $X^I$, unless there is no risk of ambiguity, in which case no superscript will be used.

  • A body (or “robot”) frame: a local reference frame fixed with respect to the robot, centered in $A$ as well. The $x$ axis points in the direction of the front of the robot, and the $y$ axis lies along the axis between the wheels, so to form a right handed reference system. We denote the robot body frame with $\{X_R, X_R\}$. The same superscript convention as above will be used. The wheels will have radius $R$.

The robot is assumed to be a rigid body, symmetric, and $X_r$ coincides with axis of symmetry. Moreover, the wheels are considered identical and at the same distance, $L$, from the axle midpoint $A$.

Moreover:

  • The center of mass $C^W = (x_c, y_c)$ of the robot is on the $x_r$ axis, at a distance $c$ from $A$, i.e., ($C^R = (c, 0)$);

  • $X^r$ forms an orientation angle $\theta$ with the local horizontal.

These notations are summarized in Figure 11.1.

Relevant notations for modeling a differential drive robot

Kinematics

✎🔗

In this section we derive the kinematic model of a differential drive mobile platform under the assumptions of (a) no lateral slipping and (b) pure rolling of the wheels. We refer to these two assumptions as kinematic constraints.

Differential drive robot kinematic constraints

✎🔗

The kinematic constraints are derived from two assumptions:

  • No lateral slipping motion: the robot cannot move sideways, but only in the direction of motion, i.e., its lateral velocity in the robot frame is zero: $$ \label{eq:mod-no-lat-slip-constraint-r}\tag{4} \dot y_A^r = 0. $$

By inverting \eqref{eq:mod-rotation-r2i}, this constraint can be expressed through the inertial frame variables, yielding:

$$ \label{eq:mod-no-lat-slip-constraint-i}\tag{5} \dot y_A(t) \cos \theta(t) -\dot x_A(t) \sin \theta(t) = 0. $$

Imposing \eqref{eq:mod-no-lat-slip-constraint-i} on \eqref{eq:mod-A-dot-polar} results in:

$$ \label{eq:mod-no-lat-slip-constraint-v_w}\tag{6} v_w(t) = \dot y_C^I(t) \cos\theta(t) - \dot x_C^I(t) \sin\theta(t), $$

and by recalling that $C^R = (c,0)$:

$$ \label{eq:mod-no-lat-slip-constraint-C}\tag{7} \dot y_C^I(t) \cos\theta(t) - \dot x_C^I(t) \sin\theta(t) = c\dot\theta(t). $$

Hence, we obtain the strongest expression of this constraint:

$$ \label{eq:mod-no-lat-slip-final}\tag{8} v_w(t) = c\dot\theta(t), $$

and therefore:

$$ \label{eq:mod-no-lat-slip-final-dot}\tag{9} \dot v_w(t) = c\ddot\theta(t). $$

a simpler way of deriving \eqref{eq:mod-no-lat-slip-final-dot} is noticing, from \eqref{eq:mod-v_A^R}, that $\dot y_A^R = v_w(t) - c\dot\theta(t)$.

  • Pure rolling: the wheels never slips or skids (Figure 11.3). Recalling that $R$ is the radius of the wheels (identical) and letting $\dot \varphi_{l}, \dot \varphi_{r}$ be the angular velocities of the left and right wheels respectively, the velocity of the ground contact point P in the robot frame is given by:
Pure rolling kinematic constraint

\begin{align} \label{eq:mod-pure-rolling}\tag{26} \left\{ \begin{array}{ll} v_{P,r} &= R \dot \varphi_{r} \\ v_{P,l} &= R \dot \varphi_{l} \end{array} \right.. \end{align}

Another notable consequence of this assumption is that, always in the robot frame, the full power of the motor can be translated into a propelling force for the vehicle in the longitudinal direction. Or, more simply, it allows to write:

$$ \label{eq:mod-force-and-torque}\tag{10} F_{u, (\cdot)}(t)R = \tau_{(\cdot)}(t), $$

where $\tau_{(\cdot)}(t)$ is the torque exerted by each motor on its wheel $(\cdot) = {l,r}$.

Differential drive robot kinematic model

✎🔗

In a differential drive robot, controlling the wheels at different speeds generates a rolling motion of rate $\omega = \dot \theta$. In a rotating field there always is a fixed point, the center of instantaneous curvature (ICC), and all points at distance $d$ from it will have a velocity given by $\omega d$, and direction orthogonal to that of the line connecting the ICC and the wheels (i.e., the axle). Therefore, by looking at Figure 11.1, we can write:

\begin{align} \label{eq:mod-kin-1}\tag{27} \left\{ \begin{array}{l} \dot \theta (d-L) &= v_l \\ \dot \theta (d+L) &= v_r \end{array} \right., \end{align}

from which:

\begin{align} \label{eq:mod-kin-2}\tag{28} \left\{ \begin{array}{l} d &= L \frac{v_r + v_l}{v_r - v_l} \\ \dot \theta &= \frac{v_r - v_l}{2L} \end{array} \right.. \end{align}

A few observations stem from \eqref{eq:mod-kin-2}:

  • If $v_r = v_l$ the bot does not turn ($\dot \theta = 0$), hence the ICC is not defined;
  • If $v_r = - v_l$, then the robot “turns on itself”, i.e., $d=0$ and $ICC \equiv A$;
  • If $v_r = 0$ (or $v_l = 0$), the rotation happens around the right (left) wheel and $d = 2L$ ($d = L$).

Moreover, a differential drive robot cannot move in the direction of the ICC, it is a singularity.

By recalling the no lateral slipping motion \eqref{eq:mod-no-lat-slip-constraint-r} hypothesis and the pure rolling constraint \eqref{eq:mod-pure-rolling}, and noticing that the translational velocity of $A$ in the robot frame is $v_A = \dot \theta d = (v_r+v_l)/2$ we can write:

\begin{align} \label{eq:mod-kin-3}\tag{29} \left\{ \begin{array}{l} \dot x_A^R &= R (\dot \varphi_R +\dot \varphi_L)/2 \\ \dot y_A^R &= 0 \\ \dot \theta &= \omega = R(\dot \varphi_R - \dot \varphi_L)/(2L) \end{array} \right., \end{align}

which in more compact yields the simplified forward kinematics in the robot frame:

\begin{align} \label{eq:mod-forward-kinematics-robot-frame}\tag{30} \left[ \begin{array}{c} \dot x_A^R \\ \dot y_A^R \\ \dot \theta \end{array} \right] = \left[ \begin{array}{cc} \frac{R}{2} & \frac{R}{2} \\ 0 & 0 \\ \frac{R}{2L} & -\frac{R}{2L} \\ \end{array} \right] \left[ \begin{array}{c} \dot \varphi_R \\ \dot \varphi_L \end{array} \right]. \end{align}

Finally, by using \eqref{eq:mod-rot-mat}, we can recast \eqref{eq:mod-forward-kinematics-robot-frame} in the inertial frame.

The simplified forward kinematics model of a differential drive vehicle is given by: \begin{align} \label{eq:mod-forward-kinematics-inertial-frame}\tag{31} \displaystyle \avec{\dot q}^I = \amat{R}(\theta) \left[ \begin{array}{c} \dot x_A^r \\ \dot y_A^r \\ \dot \theta \end{array} \right] = \left[ \begin{array}{cc} \frac{R}{2} \cos \theta & \frac{R}{2} \cos \theta \\ \frac{R}{2} \sin \theta & \frac{R}{2} \sin \theta \\ \frac{R}{2L} & -\frac{R}{2L} \\ \end{array} \right] \left[ \begin{array}{c} \dot \varphi_R \\ \dot \varphi_L \end{array} \right] = \left[ \begin{array}{cc} \cos \theta & 0 \\ \sin \theta & 0 \\ 0 & 1 \\ \end{array} \right] \left[ \begin{array}{c} v_A \\ \omega \end{array} \right]. \end{align}

Simplified dynamic model

✎🔗

By implementing the kinematic constraints formulations derived above, i.e., the no lateral slipping (\eqref{eq:mod-no-lat-slip-final-dot}) and pure rolling (\eqref{eq:mod-force-and-torque}) in the general dynamic model, it is straightforward to obtain:

\begin{align} \label{eq:mod-dyn-model-a}\tag{32} \dot{v}_u (t) &= c \dot \theta^2(t) + \frac{1}{RM} (\tau_R(t)+\tau_L(t)) \\ \dot v_w(t) &= c \dot\theta(t) \\ \ddot \theta &= - \frac{Mc}{Mc^2+J} \dot\theta(t)v_u(t) + \frac{L}{R(Mc^2+J)}(\tau_R(t)-\tau_L(t)) \end{align}

DC motor dynamic model

✎🔗

The equations governing the behavior of a DC motor are driven by an input armature voltage $V(t)$:

\begin{align} V(t) &= Ri(t) + L \frac{di}{dt} + e(t) \\ e(t) &= K_b \dot\varphi(t) \\ \tau_m(t) &= K_t i(t) \\ \tau(t) &= N \tau_m(t), \end{align}

where $(K_b, K_t)$ are the back emf and torque constants respectively and $N$ is the gear ratio ($N=1$ in the Duckiebot).

Figure 11.4 shows a diagram of a typical DC motor.

Diagram of a DC motor

Having a relation between the applied voltage and torque, in addition to the dynamic and kinematic models of a differential drive robot, allows us to determine all possible state variables of interest.

torque disturbances acting on the wheels, such as the effects of friction, can be modeled as additive terms (of sign opposite to $\tau$) in the DC motor equations.

Conclusions

✎🔗

In this chapter we derived a model for a differential drive robot. Although several simplifying assumption were made, e.g., rigid body motion, symmetry, pure rolling and no lateral slipping - still the model is nonlinear.

Regardless, we now have a sequence of descriptive tools that receive as input the voltage signal sent by the controller, and produce as output any of the state variables, e.g., the position, velocity and orientation of the robot with respect to a fixed inertial frame.

Several outstanding questions remain. For example, we need to determine what is the best representation for our robotic platform - polar coordinates, Cartesian with respect to an arbitrary reference point? Or maybe there is a better choice?

Finally, the above model assumes the knowledge of a number of constants that are characteristic of the robot’s geometry, materials, and the DC motors. Without the knowledge of those constant the model could be completely off. Determination of these parameters in a measurement driven way, i.e., the “system identification” of the robot’s plant, is subject of the odometry class.

Because of mathjax bug

Odometry Calibration

✎🔗

Jacopo

Because of mathjax bug

Computer vision basics

✎🔗

Matt

For the interested reader, there are several excellent books that discuss fundamental topics in computer vision. These include (in no particular order), Computer Vision: Algorithims and Applications (available online), Computer Vision: A Modern Approach and Multiple View Geometry in Computer Vision.

✎

Topics:

  • Objective: Discover what is present in the world, where things are, and what actions are taking place from image sequences. In the context of robotics, this corresponds to learning a (probabilistic) world model that encodes the robot’s environment, i.e., building a representation of the environment.
  • History of computer vision
    • Summer Vision Project
    • OCR (license plates, signs, checks, mail, etc.)
    • Face detection
    • Structure from motion
    • Dense reconstruction
    • Augmented reality
    • Applications to self-driving cars and beyond
Because of mathjax bug

Camera geometry

✎🔗

Matt

We could break this up into separate sub-sections for (prospective) projection and lenses.

Topics:

  • Pinhole cameras (tie into eclipse viewing, accidental cameras)
  • Geometry of projection: equations of projection; vanishing points; weak perspective; orthographic projection;
  • Review of 3D transformations (translation and rotation); homogeneous transformations
  • Perspective projection in homogeneous coordinates
✎
  • Lenses: Why?; first-order optics (thin lens); thick lens
  • First-order optics (thin lens)
Because of mathjax bug

Camera calibration

✎🔗

Matt

The discussion of intrinsic and extrinsic models could be moved up to the geometry subsection

Topics

  • Why calibration?
  • Intrinsic parameters: idealized perspective projection; pixel scaling (square and non-square); offset; skew.
  • Intrinsic model expressed in homogeneous coordinates
  • Extrinsic parameters: translation and rotation of camera frame (non-homogeneous and homogeneous)
  • Combined expression for intrinsic and extrinsic
  • Calibration targets
  • Calibration models
Because of mathjax bug

Image filtering

✎🔗

Matt

Background: Discrete-time signal processing

We currently don’t have a preliminary section on signal processing and it’s probably sufficient to point readers to a good reference (e.g., Oppenheim and Schafer).

Topics

May want to stick to linear filtering

  • Motivation (example)
  • Convolution: definition and properties (including differentiation)
  • Linear filters: examples (sharpening and smoothing)
  • Gaussian filters
    • kernels
    • properties (convolution with a Gaussian, separability)
Because of mathjax bug

Illumination invariance

✎🔗

Matt

Because of mathjax bug

Line Detection

✎🔗

Matt

Topics:

  • What is edge detection?
  • Image derivatives
  • Finite difference filters
  • Image gradients
  • Effects of noise and need for smoothing
  • Derivative theorem of convolution
  • Derivative of Gaussian filters
  • Smoothing vs. derivative filters
  • Line detection as an edge detection problem
Because of mathjax bug

Feature extraction

✎🔗

Matt

Because of mathjax bug

Place recognition

✎🔗

Matt

Because of mathjax bug

Filtering 1

✎🔗

Liam

Markov Property

Bayes Filter

Graphical representation

Kalman Filter

Extended Kalman Filter

Because of mathjax bug

Filtering 2

✎🔗

Liam

MAP estimation

Laplace Approximation

Cholesky Decomposition?

Incrementalization - Givens Rotations - iSAM

Because of mathjax bug

Mission planning

✎🔗

ETH

Because of mathjax bug

Planning in discrete domains

✎🔗

ETH

Because of mathjax bug

Motion planning

✎🔗

ETH

Because of mathjax bug

RRT

✎🔗

ETH

Because of mathjax bug

Feedback control

✎🔗

Jacopo

Because of mathjax bug

PID Control

✎🔗

Jacopo

Because of mathjax bug

MPC Control

✎🔗

Jacopo

Because of mathjax bug

Object detection

✎🔗

Nick and David

Because of mathjax bug

Object classification

✎🔗

Nick and David

Because of mathjax bug

Object tracking

✎🔗

Nick and David

Because of mathjax bug

Reacting to obstacles

✎🔗

Jacopo

Because of mathjax bug

Semantic segmentation

✎🔗

Nick and David

Because of mathjax bug

Text recognition

✎🔗

Nick

Because of mathjax bug

SLAM - Problem formulation

✎🔗

Liam

Because of mathjax bug

SLAM - Broad categories

✎🔗

Liam

Because of mathjax bug

VINS

✎🔗

Liam

Because of mathjax bug

Advanced place recognition

✎🔗

Liam

Because of mathjax bug

Fleet level planning (placeholder)

✎🔗

ETH

Because of mathjax bug

Fleet level planning (placeholder)

✎🔗

ETH

Because of mathjax bug

Bibliography

✎🔗
Liam Paull, Jacopo Tani, Heejin Ahn, Javier Alonso-Mora, Luca Carlone, Michal Cap, Yu Fan Chen, Changhyun Choi, Jeff Dusek, Daniel Hoehener, Shih-Yuan Liu, Michael Novitzky, Igor Franzoni Okuyama, Jason Pazis, Guy Rosman, Valerio Varricchio, Hsueh-Cheng Wang, Dmitry Yershov, Hang Zhao, Michael Benjamin, Christopher Carr, Maria Zuber, Sertac Karaman, Emilio Frazzoli, Domitilla Del Vecchio, Daniela Rus, Jonathan How, John Leonard, and Andrea Censi. Duckietown: an open, inexpensive and flexible platform for autonomy education and research. In IEEE International Conference on Robotics and Automation (ICRA). Singapore, May 2017. pdf supp. materialbibtex Bruno Siciliano and Oussama Khatib. Springer Handbook of Robotics. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2007.    Jacopo Tani, Liam Paull, Maria Zuber, Daniela Rus, Jonathan How, John Leonard, and Andrea Censi. Duckietown: an innovative way to teach autonomy. In EduRobotics 2016. Athens, Greece, December 2016. pdf supp. materialbibtex Tosini, G., Ferguson, I., Tsubota, K. Effects of blue light on the circadian system and eye physiology. Molecular Vision, 22, 61–72, 2016 (online). Albert Einstein. Zur Elektrodynamik bewegter Körper. (German) On the electrodynamics of moving bodies. Annalen der Physik, 322(10):891–921, 1905.    DOI  Ivan Savov. Linear algebra explained in four pages. https://minireference.com/static/tutorials/linear_algebra_in_4_pages.pdf, 2017. Online; accessed 23 August 2017.    Kaare Brandt Petersen and Michael Syskind Pedersen. The matrix cookbook.    .pdf  Matrix (mathematics). Matrix (mathematics) — Wikipedia, the free encyclopedia, 2017. Online; accessed 2-September-2017.    www:  Peter D. Lax. Functional Analysis. Wiley Interscience, New York, NY, USA, 2002.    Athanasios Papoulis and S. Unnikrishna Pillai. Probability, Random Variables and Stochastic Processes. McGraw Hill, fourth edition, 2002.    David J. C. MacKay. Information Theory, Inference & Learning Algorithms. Cambridge University Press, New York, NY, USA, 2002.    H. Choset, W. Burgard, S. Hutchinson, G. Kantor, L. E. Kavraki, K. Lynch, and S. Thrun. Principles of robot motion: Theory, agorithms, and implementation. MIT Press, June 2005.    Steven M. LaValle. Planning Algorithms. Cambridge University Press, New York, NY, USA, 2006.    M. Miskowicz. Event-Based Control and Signal Processing. Embedded Systems. CRC Press, 2015.    http  Karl J. Aström. Event Based Control, pages 127–147. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.    DOI  http 
Summary. In spite of the success of traditional sampled-data theory in computer control it has some disadvantages particularly for distributed, asynchronous, and multi-rate system. Event based sampling is an alternative which is explored in this paper. A simple example illustrates the differences between periodic and event based sampling. The architecture of a general structure for event based control is presented. The key elements are an event detector, an observer, and a control signal generator, which can be regarded as a generalized data-hold. Relations to nonlinear systems are discussed. Design of an event based controller is illustrated for a simple model of a micro-mechanical accelerometer.

Edward A Lee and David G Messerschmitt. Synchronous data flow. Proceedings of the IEEE, 75(9):1235–1245, 1987.    Rajit Manohar and K Mani Chandy. Delta-dataflow networks for event stream processing. pages 1–6, June 2010.    Romano M DeSantis. Modeling and path-tracking control of a mobile wheeled robot with a differential drive. Robotica, 13(4):401–410, 1995.    Richard Szeliski. Computer Vision: Algorithms an Applications. Springer, 2010.    http  David Forsyth and Jean Ponce. Computer Vision: A Modern Approach. Prentce Hall, Upper Saddle River, NJ, 2011.    Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004.   
Because of mathjax bug

Exercises

✎🔗

These exercises can guide you from the status of a novice coder to experienced roboticist.

Because of mathjax bug

Exercise: Basic image operations

✎🔗

Andrea Daniele

Exercise: Basic image operations, adult version

✎🔗

Andrea Daniele

dt-image-flip specification

✎🔗

The program image-ops expects exactly two arguments: a filename (a JPG file) and a directory name.

$ dt-image-flip file outdir

If the file does not exist, the script must exit with error code 2.

If the file cannot be decoded, the script must exit with error code 3.

If the file exists, then the script must create:

  • outdir/regular.jpg: a copy of the initial file
  • outdir/flip.jpg: the file, flipped vertically.
  • outdir/side-by-side.jpg: the two files, side by side.

If any other error occurs, the script should exit with error code 99.

The original picture.
The output flip.jpg
The output side-by-side.jpg
Example input-output for the program image-ops.

Testing it works with image-ops-tester

✎🔗

We provide 4 scripts that can be used to make sure that you wrote a conforming dt-image-flip. The scripts are image-ops-tester-good, image-ops-tester-bad1, image-ops-tester-bad2, and image-ops-tester-bad3. You can find them in the directory /exercises/dt-image-flip/image-ops-tester in the duckietown/duckuments repository.

The script called image-ops-tester-good tests your program in a situation in which we expect it to work properly. The 3 “bad” test scripts (i.e., image-ops-tester-bad1 through image-ops-tester-bad3) test your code in situations in which we expect your program to complain in the proper way.

Use them as follows:

$ image-ops-tester-scenario candidate-program

The tester scripts must be called from their own location. Make sure to change your working directory to /exercises/dt-image-flip/image-ops-tester before launching the tester scripts.

If the script cannot be found, image-ops-tester-scenario will return 1.

image-ops-tester-scenario will return 0 if the program exists and conforms to the specification (Section 2.3 - dt-image-flip specification).

If it can establish that the program is not good, it will return 11.

Exercise: Simple data analysis from a bag

✎🔗

Andrea Daniele

Exercise: Bag in, bag out

✎🔗

Andrea Daniele

Exercise: Bag thumbnails

✎🔗

Andrea Daniele

Exercise: Instagram filters

✎🔗

Andrea Daniele

Exercise: Bag instagram

✎🔗

Andrea Daniele

Exercise: Live Instagram

✎🔗

Andrea Daniele

Exercise: Augmented Reality

✎🔗

Jonathan Michaux and Dzenan Lapandic

Specification of dt_augmented_reality

✎🔗

In this assignment you will be writing a ROS package to perform the augmented reality exercise. The program will be invoked with the following syntax:

$ roslaunch dt_augmented_reality-robot name augmenter.launch map_file:=map file robot_name:=robot name local:=1

where map file is a YAML file containing the map (specified in Section 9.5 - Specification of the map).

If robot name is not given, it defaults to the hostname.

The program does the following:

  1. It loads the intrinsic / extrinsic calibration parameters for the given robot.
  2. It reads the map file.
  3. It listens to the image topic /robot name/camera_node/image/compressed.

  4. It reads each image, projects the map features onto the image, and then writes the resulting image to the topic

    /![robot name]/AR/![map file basename]/image/compressed

where map file basename is the basename of the file without the extension.

We provide you with ROS package template that contains the AugmentedRealityNode. By default, launching the AugmentedRealityNode should publish raw images from the camera on the new /robot name/AR/map file basename/image/compressed topic.

In order to complete this exercise, you will have to fill in the missing details of the Augmenter class by doing the following:

  1. Implement a method called process_image that undistorts raw images.
  2. Implement a method called ground2pixel that transforms points in ground coordinates (i.e. the robot reference frame) to pixels in the image.
  3. Implement a method called callback that writes the augmented image to the appropriate topic.

Specification of the map

✎🔗

The map file contains a 3D polygon, defined as a list of points and a list of segments that join those points.

The format is similar to any data structure for 3D computer graphics, with a few changes:

  1. Points are referred to by name.
  2. It is possible to specify a reference frame for each point. (This will help make this into a general tool for debugging various types of problems).

Here is an example of the file contents, hopefully self-explanatory.

The following map file describes 3 points, and two lines.

points:
    # define three named points: center, left, right
    center: [axle, [0, 0, 0]] # [reference frame, coordinates]
    left: [axle, [0.5, 0.1, 0]]
    right: [axle, [0.5, -0.1, 0]]
segments:
- points: [center, left]
  color: [rgb, [1, 0, 0]]
- points: [center, right]
  color: [rgb, [1, 0, 0]]

Exercise: Lane Filtering - Particle Filter

✎🔗

Jonathan Michaux, Liam Paull, and Miguel de la Iglesia

Exercise: Lane Filtering - Extended Kalman Filter

✎🔗

Liam Paull

Exercise: Git and conventions

✎🔗

move here the exercises we had last year about branching.

Exercise: Use our API for arguments

✎🔗

Exercise: Bouncing ball

✎🔗

Exercise: Visualizing data on image

✎🔗

Exercise: Make that into a node

✎🔗

Exercise: Instagram with EasyAlgo interface

✎🔗

Milestone: Illumination invariance (anti-instagram)

✎🔗

Make them run our code, and also visualize what’s going on

Exercise: Launch files basics

✎🔗

Exercise: Unit tests

✎🔗

Exercise: Parameters (standard ROS api)

✎🔗

Exercise: Parameters (our API)

✎🔗

Exercise: Place recognition abstraction

✎🔗

Exercise: Parallel processing

✎🔗

Exercise: Adding new test to integration tests

✎🔗

to write

Exercise: localization

✎🔗

to write

Exercise: Ducks in a row

✎🔗

to write

Exercise: Comparison of PID

✎🔗

to write

Exercise: RRT

✎🔗

to write

Because of mathjax bug

Exercise: Who watches the watchmen? (optional)

✎🔗

Software reference

✎🔗

This part describes things that you should know about UNIX/Linux environments.

Documentation writers: please make sure that every command used has a section in these chapters.

Because of mathjax bug

Ubuntu packaging with APT

✎🔗

apt-key

✎🔗

to write

apt-mark

✎🔗

to write

apt-get

✎🔗

to write

add-apt-repository

✎🔗

to write

wajig

✎🔗

to write

dpigs

✎🔗

to write

Because of mathjax bug

GNU/Linux general notions

✎🔗

Andrea

Every day Linux

✎🔗

man

✎🔗

This is an interface to the on-line reference manuals. Whenever you meet some unfamiliar commands, try use man certain_command before Googling. You will find it extremely clear, useful and self-contained.

cd

✎🔗

Go to the directory you want. If you just use:

laptop $ cd

then you will go to your home directory, i.e., ~

sudo

✎🔗

Whenever you want to modify system files, you will need sudo. Commonly touched system files including /etc, /opt and so on. Since most of you have followed the guideline to use passwordless sudo, I would recommend that make sure what you are doing with sudo before you execute the command, otherwise you may need to reinstall the system.

ls

✎🔗

List all the files and documents in the current directory. From ~/.bashrc, we know some commonly used alias. See more by man ls.

– la for ls -a which will list out all files and documents including the hidden ones(whose name starts with a dot).

– ll for ls -l which will display Unix file types, permissions, number of hard links, owner, group, size, last-modified date and filename.

cp

✎🔗

cp fileA directoryB will copy the file A to directory B. See more by executing man cp.

mkdir

✎🔗

Make new directory. See more by man mkdir.

touch

✎🔗

Update the access and modification times of the input file to current time. See more by man touch.

reboot

✎🔗

This command must be executed as root. sudo required. This will reboot your laptop or Raspberry Pi. See more by man reboot.

shutdown

✎🔗

This command requires sudo. You can set a countdown to shutdown you machine. More by man shutdown.

rm

✎🔗

Remove certain file. rm -r will remove files. More in man rm.

Users

✎🔗

passwd

✎🔗

Update password of the current user. Old password needed.

UNIX tools

✎🔗

cat

✎🔗

Cat some file will return you the content. More in man cat.

tee

✎🔗

Read from standard input and write to standard output and files. More on man tee.

truncate

✎🔗

to write

Linux disks and files

✎🔗

fdisk

✎🔗

to write

mount

✎🔗

to write

umount

✎🔗

to write

losetup

✎🔗

to write

gparted

✎🔗

to write

dd

✎🔗

to write

sync

✎🔗

to write

df

✎🔗

to write

How to make a partition

✎🔗

to write

Other administration commands

✎🔗

visudo

✎🔗

to write

update-alternatives

✎🔗

to write

udevadm

✎🔗

to write

systemctl

✎🔗

to write

Make

✎🔗

make

✎🔗

to write

Python-related tools

✎🔗

virtualenv

✎🔗

to write

pip

✎🔗

to write

Raspberry-PI commands

✎🔗

raspi-config

✎🔗

to write

vcgencmd

✎🔗

to write

raspistill

✎🔗

to write

jstest

✎🔗

to write

swapon

✎🔗

to write

mkswap

✎🔗

to write

Users and permissions

✎🔗

chmod

✎🔗

to write

groups

✎🔗

to write

adduser

✎🔗

to write

useradd

✎🔗

to write

Downloading

✎🔗

curl

✎🔗

to write

wget

✎🔗

to write

sha256sum

✎🔗

to write

xz

✎🔗

to write

Shells and environments

✎🔗

source

✎🔗

You can only do source file_name if the file can be executed by bash.

which

✎🔗

Tell you the /bin/ directory of your command. This is useful to distinguish which python you are using if you have virtualenv.

export

✎🔗

to write

Other misc commands

✎🔗

npm

✎🔗

to write

nodejs

✎🔗

to write

ntpdate

✎🔗

to write

chsh

✎🔗

to write

echo

✎🔗

to write

sh

✎🔗

to write

fc-cache

✎🔗

to write

Mounting USB drives

✎🔗

First plug in the USB drive nothing will work if you don’t do that first. Now ssh into your robot. On the command line type:

duckiebot $ lsusb

you should see your Sandisk USB drive as an entry. Congrats, you correctly plugged it in

duckiebot $ lsblk

Under name you should see sda1, with size about 28.7GB and nothing under the MOUNTPOINT column (if you see something under MOUNTPOINT congrats you are done.

Next make a directory to mount to:

duckiebot $ sudo mkdir /media/logs

Next mount the drive

duckiebot $ sudo mount -t vfat /dev/sda1 /media/logs -o umask=000

Test by running lsblk again and you should now see /media/logs under MOUNTPOINT

Unmounting a USB drive

✎🔗
duckiebot $ sudo umount /media/logs
Because of mathjax bug

Linux resources usage

✎🔗

SD Cards tools

✎🔗

Testing SD Card and disk speed

✎🔗

Test SD Card (or any disk) speed using the following commands, which write to a file called filename.

$ dd if=/dev/zero of=filename bs=500K count=1024
$ sync
$ echo 3 | sudo tee /proc/sys/vm/drop_caches
$ dd if=filename of=/dev/null bs=500K count=1024
$ rm filename

Note the sync and the echo command are very important.

Example results:

524288000 bytes (524 MB, 500 MiB) copied, 30.2087 s, 17.4 MB/s
524288000 bytes (524 MB, 500 MiB) copied, 23.3568 s, 22.4 MB/s

That is write 17.4 MB/s, read 22 MB/s.

How to burn an image to an SD card

✎🔗

A blank SD card.

An image file to burn.

An Ubuntu computer with an SD reader.

A burned image.

How to shrink an image

✎🔗

An image file to burn.

An Ubuntu computer.

A shrunk image.

Majority of content taken from here

We are going to use the tool gparted so make sure it’s installed

laptop $ sudo apt install gparted

Let the image file be image file and its name be imagename. Run the command:

laptop $ sudo fdisk -l image file

It should give you something like:

Device                       Boot  Start      End  Sectors  Size Id Type
duckiebot-RPI3-LP-aug15.img1        2048   131071   129024   63M  c W95 FAT32 (LBA)
duckiebot-RPI3-LP-aug15.img2      131072 21219327 21088256 10.1G 83 Linux

Take note of the start of the Linux partition (in our case 131072), let’s call it start. Now we are going to mount the Linux partition from the image:

laptop $ sudo losetup /dev/loop0 imagename.img -o $((start*512))

and then run gparted:

laptop $ sudo gparted /dev/loop0

In gparted click on the partition and click “Resize” under the “Partition” menu. Resize drag the arrow or enter a size that is equal to the minimum size plus 20MB

This didn’t work well for me - I had to add much more than 20MB for it to work.

Click the “Apply” check mark. Before closing the final screen click through the arrows in the dialogue box to find a line such a “resize2fs -p /dev/loop0 1410048K”. Take note of the new size of your partition. Let’s call it new size.

Now remove the loopback on the second partition and setup a loopback on the whole image and run fdisk:

laptop $ sudo losetup -d /dev/loop0
laptop $ sudo losetup /dev/loop0 image file
laptop $ sudo fdisk /dev/loop0

Command (m for help): enter d
Partition number (1,2, default 2): enter 2
Command (m for help): enter n
Partition type
p   primary (1 primary, 0 extended, 3 free)
e   extended (container for logical partitions)
Select (default p): enter p
Partition number (2-4, default 2): enter 2
First sector (131072-62521343, default 131072): start
Last sector, +sectors or +size{K,M,G,T,P} (131072-62521343, default 62521343): +new sizeK

on the last line include the + and the K as part of the size.

Created a new partition 2 of type 'Linux' and of size 10.1 GiB.

Command (m for help): enter w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument

The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).

Disregard the final error.

You partition has now been resized and the partition table has been updated. Now we will remove the loopback and then truncate the end of the image file:

laptop $ fdisk -l /dev/loop0


Device       Boot  Start      End  Sectors  Size Id Type
/dev/loop0p1        2048   131071   129024   63M  c W95 FAT32 (LBA)
/dev/loop0p2      131072 21219327 21088256 10.1G 83 Linux

Note down the end of the second partition (in this case 21219327). Call this end.

laptop $ sudo losetup -d /dev/loop0
laptop $ sudo truncate -s $(((end+1)*512)) image file

You now have a shrunken image file.

It might be useful to compress it, before distribution:

laptop $ xz image file
Because of mathjax bug

Networking tools

✎🔗

Andrea

Preliminary reading:

  • Basics of networking, including

    • what are IP addresses
    • what are subnets
    • how DNS works
    • how .local names work
    • …

XXX (ref to find).

to write

Make sure that you know:

hostname

✎🔗

to write

Accessing computers using SSH

✎🔗

Andrea

Local configuration

✎🔗

The SSH configuration as a client is in the file

~/.ssh/config

Create the directory with the right permissions:

$ mkdir ~/.ssh
$ chmod 0700 ~/.ssh

Edit the file:

~/.ssh/config

(We suggest you use VIM to edit files; see a tutorial here.)

comment

laptop or duckiebot? - LP

Then add the following lines:

HostKeyAlgorithms ssh-rsa

The reason is that Paramiko, used by roslaunch, does not support the ECSDA keys.

How to login with SSH and a password

✎🔗

To log in to a remote computer remote with user remote-user, use:

$ ssh remote-user@remote

Creating an SSH keypair

✎🔗

This is a step that you will repeat twice: once on the Duckiebot, and once on your laptop.

The program will prompt you for the filename on which to save the file.

Use the convention

/home/username/.ssh/username@host name
/home/username/.ssh/username@host name.pub

where:

  • username is the current user name that you are using (ubuntu or your chosen one);
  • host name is the name of the host (the Duckiebot or laptop);

An SSH key can be generated with the command:

$ ssh-keygen -h

The session output will look something like this:

Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):

At this point, tell it to choose this file:

/home/username/.ssh/username@host name

Then:

Enter passphrase (empty for no passphrase):

Press enter; you want an empty passphrase.

Enter same passphrase again:

Press enter.

Your identification has been saved in /home/username/.ssh/username@host name
Your public key has been saved in /home/username/.ssh/username@host name.pub
The key fingerprint is:
XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX username@host name
The key's randomart image is:
+--[ RSA 2048]----+
|            .    |
|       o   o  .  |
|      o = o  . o |
|       B . .  * o|
|        S o    O |
|         o o  . E|
|          o o  o |
|           o  +  |
|            .. . |
+-----------------+

Note that the program created two files.

The file that contains the private key is

/home/username/.ssh/username@host name

The file that contains the public key has extension .pub:

/home/username/.ssh/username@host name.pub

Next, tell SSH that you want to use this key.

Make sure that the file ~/.ssh/config exists:

$ touch ~/.ssh/config

Add a line containing

IdentityFile ~/.ssh/PRIVATE_KEY_FILE

(using the filename for the private key).

comment

make sure to include the full path to the file, not just the filename.

Check that the config file is correct:

$ cat ~/.ssh/config
...
IdentityFile ~/.ssh/PRIVATE_KEY_FILE
...

To copy the generated SSH key to the clipboard xclip can be used (Installation of xclip if necessary).

$ sudo apt-get install xclip
$ xclip -sel clip < ~/.ssh/username@host name.pub

How to login without a password

✎🔗

You have two computers, called “local” and “remote”, with users “local-user” and “remote-user”. Here, we assume that local and remote are complete hostnames (such as duckiebot.local.).

The two computers are on the same network and they can ping each other.

You have created a keypair for local-user on local. This procedure is described in Section 19.5 - Creating an SSH keypair.

From the local computer, local-user will be able to log in to remote computer without a password.

First, connect the two computers to the same network, and make sure that you can ping remote from local:

local $ ping remote.local

Do not continue if you cannot do this successfully.

If you have created a keypair for local-user, you will have a public key in this file on the local computer:

/home/local-user/.ssh/local-user@local.pub

This file is in the form:

ssh-rsa long list of letters and numbers local-user@local

You will have to copy the contents of this file on the remote computer, to tell it that this key is authorized.

On the local computer, run the command:

local $ ssh-copy-id remote-user@remote

now you should be able to login to the remote without a password:

local $ ssh remote-user@remote

This should succeed, and you should not be asked for a password.

Fixing SSH Permissions

✎🔗

Sometimes, SSH does not work because you have the wrong permissions on some files.

In doubt, these lines fix the permissions for your .ssh directory.

$ chmod 0700 ~/.ssh
$ chmod 0700 ~/.ssh/*

ssh-keygen

✎🔗

to write

Because of mathjax bug

Wireless networking in Linux

✎🔗

iwconfig

✎🔗

to write

iwlist

✎🔗

Moving files between computers

✎🔗

scp

✎🔗

to write

Download a file with SCP

✎🔗

Use this command:

laptop $ scp hostname:/path/to/out.jpg .

to download out.jpg to your current directory.

rsync

✎🔗

to write

Because of mathjax bug

VIM

✎🔗

Andrea

To do quick changes to files, especially when logged remotely, we suggest you use the VI editor, or more precisely, VIM (“VI iMproved”).

vi

✎🔗

to write

Atom

✎🔗

Install Atom

✎🔗

Following the instructions here:

$ sudo add-apt-repository ppa:webupd8team/atom
$ sudo apt update
$ sudo apt install atom

Using Atom to code remotely

✎🔗

With Atom, you are able to remotely code on files located on your Duckiebot with a GUI. The benefit of using Atom is that you are able to install extensions such as an IDE for Python, a Markdown previewer, or just use custom themes to avoid coding in the terminal.

Follow these instructions:

Install remote-atom

laptop $ sudo apm install remote-atom

Now, we need to edit our SSH config so that any data send to the port 52698, which is the port remote-atom is using, is forwarded via SSH to our local machine. Edit the file “~/.ssh/config”. There, you add “RemoteForward 52698 127.0.0.1:52698” to your host. The resulting host will look similar to

Host lex
    User julien
    Hostname lex.local
    RemoteForward 52698 127.0.0.1:52698

Now, we need to connect to our duckiebot via SSH and install rmate (and simultaniously rename it to ratom)

duckiebot $ sudo wget -O /usr/local/bin/ratom https://raw.github.com/aurora/rmate/master/rmate
duckiebot $ sudo chmod +x /usr/local/bin/ratom

Now, you just need to launch Atom on your local machine, go to Packages->Remote Atom->Start Server.

You can now edit a file in a terminal connected to your duckiebot via SSH by typing

duckiebot $ sudo ratom filename

And atom will automatically open the file on your local machine. In the settings of remote-atom, you can also set the package to start the server automatically on launching atom on your local machine.

Because of mathjax bug

Liclipse

✎🔗

Installing LiClipse

✎🔗

Follow the instructions at this page.

At the moment of writing, these are:

$ wget http://www.mediafire.com/file/rwc4bk3nthtxcvv/liclipse_4.1.1_linux.gtk.x86_64.tar.gz
$ tar xvzf liclipse_4.1.1_linux.gtk.x86_64.tar.gz
$ sudo ln -s `pwd`/liclipse/LiClipse /usr/bin/liclipse

Now you can run it using liclipse:

$ liclipse

When it runs for the first time, choose “use this as default” and click “launch”.

Choose “Import-> General -> Existing project into workspace”. Select the folder ~/duckietown.

comment

Only Import -> General -> Projects from Folder or Archive, selecting ~/duckuments worked for me. JT

comment

This is not about the duckuments, it’s for duckietown - AC

If it asks about interpreters, select “auto config”.

When it shows “uncheck settings that should not be changed”, just click OK.

Slack

✎🔗

Disabling Slack email notifications

✎🔗

Most importantly, please take the time to disable email notification for Slack.

The point of Slack is that you don’t get email. You can work on Duckietown only when you have the time to do so.

to write how

Disabling Slack pop-up notification on the desktop

✎🔗

Also remove pop up notifications from the app. It should be a discrete notification that says “hey, when you have some time, look at Twist”, not “HEY HEY HEY PAY ATTENTION TO ME NOW”.

to write procedure

Because of mathjax bug

Byobu

✎🔗

Andrea

You need to learn to use Byobu. It will save you much time later.

(Alternatives such as GNU Screen are fine as well.)

Source code control with Git

✎🔗

Andrea

Setting up global configurations for Git

✎🔗

This should be done twice, once on the laptop, and later, on the robot.

These options tell Git who you are:

$ git config --global user.email "email"
$ git config --global user.name  "full name"

Also do this, and it doesn’t matter if you don’t know what it is:

$ git config --global push.default simple

git

✎🔗

to write

hub

✎🔗

Git LFS

✎🔗

This describes Git LFS.

Generic installation instructions

✎🔗

See instructions at:

https://git-lfs.github.com/

Troubleshooting

✎🔗

The binary files are not downloaded. In their place, there are short “pointer” files.

If you have installed LFS after pulling the repository and you see only the pointer files, do:

$ git lfs pull --all
Because of mathjax bug

Setup Github access

✎🔗

Andrea

This chapter describes how to create a Github account and setup SSH on the robot and on the laptop.

Add a public key to Github

✎🔗

You will do this procedure twice: once for the public key created on the laptop, and later with the public key created on the robot.

A public/private keypair already created and configured. This procedure is explained in Section 19.5 - Creating an SSH keypair.

You can access Github using the key provided.

comment

Since I am not as familiar with Linux an VIM it would have been great to say how we can get access to the public key: sudo vim /home/username/.ssh/username@host name.pub and than copy it and add it on github SL

Go to settings (Figure 29.2).

Add the public key that you created:

To check that all of this works, use the command

$ ssh -T git@github.com

The command tries to connect to Github using the private keys that you specified. This is the expected output:

Warning: Permanently added the RSA host key for IP address 'ip address' to the list of known hosts.
Hi username! You've successfully authenticated, but GitHub does not provide shell access.

If you don’t see the greeting, stop.

Repeat what you just did for the Duckiebot on the laptop as well, making sure to change the name of the file containing the private key.

Because of mathjax bug

ROS installation and reference

✎🔗

Liam

Install ROS

✎🔗

This part installs ROS. You will run this twice, once on the laptop, once on the robot.

The first commands are copied from this page.

Tell Ubuntu where to find ROS:

$ sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'

Tell Ubuntu that you trust the ROS people (they are nice folks):

$ sudo apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116

Fetch the ROS repo:

$ sudo apt update

Now install the mega-package ros-kinetic-desktop-full.

$ sudo apt install ros-kinetic-desktop-full

There’s more to install:

$ sudo apt install ros-kinetic-{tf-conversions,cv-bridge,image-transport,camera-info-manager,theora-image-transport,joy,image-proc,compressed-image-transport,phidgets-drivers,imu-complementary-filter,imu-filter-madgwick}

Do not install packages by the name of ros-X, only those by the name of ros-kinetic-X. The packages ros-X are from another version of ROS.

XXX: not done in aug20 image:

Initialize ROS:

$ sudo rosdep init
$ rosdep update

rqt_console

✎🔗

to write

roslaunch

✎🔗

to write

rviz

✎🔗

to write

rostopic

✎🔗

to write

catkin_make

✎🔗

to write

rosrun

✎🔗

to write

rostest

✎🔗

to write

rospack

✎🔗

to write

rosparam

✎🔗

to write

rosdep

✎🔗

to write

roswtf

✎🔗

to write

rosbag

✎🔗

A bag is a file format in ROS for storing ROS message data. Bags, so named because of their .bag extension, have an important role in ROS. Bags are typically created by a tool like rosbag, which subscribe to one or more ROS topics, and store the serialized message data in a file as it is received. These bag files can also be played back in ROS to the same topics they were recorded from, or even remapped to new topics.

roscore

✎🔗

to write

Troubleshooting ROS

✎🔗

computer is not in your SSH known_hosts file

See this thread. Remove the known_hosts file and make sure you have followed the instructions in Section 19.3 - Local configuration.

How to install PyTorch on the Duckiebot

✎🔗

PyTorch is a Python deep learning library that’s currently gaining a lot of traction, because it’s a lot easier to debug and prototype (compared to TensorFlow / Theano).

To install PyTorch on the Duckietbot you have to compile it from source, because there is no pro-compiled binary for ARMv7 / ARMhf available. This guide will walk you through the required steps.

How to install Caffe and Tensorflow on the Duckiebot

✎🔗

Caffe and TensorFlow are popular deep learning libraries, and are supported by the Intel Neural Computing Stick (NCS).

Movidius Neural Compute Stick Install

✎🔗

How To Use Neural Compute Stick

✎🔗

Software development guide

✎🔗

This part is about how to develop software for the Duckiebot.

Because of mathjax bug

Python

✎🔗

Context managers

✎🔗

to write

Working with YAML files

✎🔗

YAML files are useful for writing configuration files, and are used a lot in Duckietown.

Duckietown code conventions

✎🔗

Python

✎🔗

Tabs

✎🔗

Never use tabs in Python files.

The tab characters are evil in Python code. Please be very careful in changing them.

Do not use a tool to do it (e.g. “Convert tabs to spaces”); it will get it wrong.

checked by what-the-duck.

Configuration

✎🔗

This chapter explains what are the assumptions about the configuration.

While the “Setup” parts are “imperative” (do this, do that); this is the “declarative” part, which explains what are the properties of a correct configuration (but it does not explain how to get there).

The tool what-the-duck (Subsection 8.1.3 - The what-the-duck program) checks some of these conditions. If you make a change from the existing conditions, make sure that it gets implemented in what-the-duck by filing an issue.

Environment variables (updated Sept 12)

✎🔗

You need to have set up the variables in Table 4.1.

The way to set these up is to add them in the file ~/.bashrc (export var="value"). Do not modify the environment.sh script.

Environment variables used by the software
variable reasonable value contains
DUCKIETOWN_ROOT ~/duckietown Software repository
DUCKIEFLEET_ROOT ~/duckiefleet Where to look for class-specific information (people DB, robots DB).
DUCKIETOWN_DATA ~/duckietown-data The place where to look for logs.
DUCKIETOWN_TMP If set, directory to use for temporary files. If not set, we use the default ( /tmp).
DUCKIETOWN_CONFIG_SEQUENCE defaults:baseline:vehicle:user The configuration sequence for EasyNode

Duckietown root directory DUCKIETOWN_ROOT

✎🔗

to write

Duckiefleet directory DUCKIEFLEET_ROOT

✎🔗

For Fall 2017, this is the the repository duckiefleet.

For self-guided learners, this is an arbitrary repository to create.

The “scuderia” (vehicle database)

✎🔗

The system needs to know certain details about the robots, such as their host names, the name of the users, etc.

This data is contained in the ${DUCKIEFLEET_ROOT}/robots/{your_branch} directory, in files with the pattern robot name.robot.yaml.

The file must contain YAML entries of the type:

owner: ID of owner
username: username on the machine
hostname: host name
description: generic description
log:
    date: comment
    date: comment

A minimal example is in Listing 4.6.

owner: censi
hostname: emma
username: andrea
description: Andrea's car.
log:
    2017-08-01: >
        Switched RPI2 with RPI3.
    2017-08-20: >
        There is something wrong with the PWM hat and the LEDs.
Minimal scuderia file emma.robot.yaml

Explanations of the fields:

  • hostname: the host name. This is normally the same as the robot name.
  • username: the name of the Linux user on the robot, from which to run programs.
  • owner: the owner’s globally-unique Duckietown ID.

The machines file

✎🔗

Make sure you already set up ROS (Section 11.3 - Set up the ROS environment on the Duckiebot).

Activate ROS:

$ cd ~/duckietown
$ source environment.sh

The machines file is created from the scuderia data using this command:

$ rosrun duckieteam create-machines

People database

✎🔗

Andrea

Describe the people database.

Modes of operation

✎🔗

There are 3 modes of operation:

  1. MODE-normal: Everything runs on the robot.
  2. MODE-offload: Drivers run on the robot, but heavy computation runs on the laptop.
  3. MODE-bag: The data is provided from a bag file, and computation runs on the laptop.
Operation modes
mode name who is the ROS master where data comes from where heavy computation happen
MODE-normal duckiebot Drivers on Duckiebot duckiebot
MODE-offload duckiebot Drivers on Duckiebot laptop
MODE-bag laptop Bag file laptop
Because of mathjax bug

Node configuration mechanisms

✎🔗

Where the config files are, how they are used.

Because of mathjax bug

Minimal ROS node - pkg_name

✎🔗

Andrea

This document outline the process of writing a ROS package and nodes in Python.

To follow along, it is recommend that you duplicate the pkg_name folder and edit the content of the files to make your own package.

Launch File

✎🔗

You should always write a launch file to launch a node. It also serves as a documentation on the I/O of the node.

Let’s take a look at launch/test.launch.

<launch>
    <node name="talker" pkg="pkg_name" type="talker.py" output="screen">

        <param name="~pub_timestep" value="0.5"/>

        <remap from="~topic_b" to="~topic_a"/>
    </node>
</launch>

For the <node>, the name specify the name of the node, which overwrites rospy.init_node() in the __main__ of talker.py. The pkg and type specify the package and the script of the node, in this case it’s talker.py.

Don’t forget the .py in the end (and remember to make the file executable through chmod).

The output="screen" direct all the rospy.loginfo to the screen, without this you won’t see any printouts (useful when you want to suppress a node that’s too talkative.)

The <param> can be used to set the parameters. Here we set the ~pub_timestep to 0.5. Note that in this case this sets the value of talker/pub_timestep to 0.5.

The <remap> is used to remap the topic names. In this case we are replacing ~topic_b with ~topic_a so that the subscriber of the node actually listens to its own publisher. Replace the line with

<remap from="~topic_b" to="talker/topic_a"/>

will have the same effect. This is redundant in this case but very useful when you want to subscribe to a topic published by another node.

Makefile system

✎🔗

Andrea

We use Makefiles to describe frequently-used commands.

Jupyter

✎🔗

Installation

✎🔗

Pull the branch 1710-place-recognition

$ cd duckietown
$ git pull

Source environment:

$ source environment.sh

Remove previous installations:

$ sudo apt remove ipython ipython-notebook

Then run:

$ pip install --user jupyter IPython==5.0

Check the versions are correct:

$ which ipython

/home/andrea/.local/bin/ipython

Check the version is correct:

$ ipython --version

5.0.0

ROS package verification

✎🔗

Andrea

This chapter describes formally what makes a conforming ROS package in the Duckietown software architecture.

Duckietown utility library

✎🔗

Bug squashing guide

✎🔗

This unit describes how to debug your programs.

Do read this accurately top-to-bottom. If you think this is too long and too verbose to read and you are in a hurry anyway: that is probably the attitude that introduced the bug.

The basic truths of bug squashing

✎🔗

What could it be?

✎🔗

How to find the bug by yourself

✎🔗

Step 0: Is it late? Go to bed.

✎🔗

If it is later than 10pm, just go to bed, and look at it tomorrow.

After 10pm, bugs are introduced, rather than removed.

Step 1: Are you in a hurry? Do it another time.

✎🔗

Bug squashing requires a clear mind.

If you are in a hurry, it’s better you do this another time; otherwise, you will not find the bug and you will only grow more frustrated.

Step 2: Make sure your environment is sane using what-the-duck

✎🔗

Finding a bug is a process of elimination of causes one-by-one, until you find the real culprit. Most of the problems come ultimately from the fact that your environment is not set up correctly.

We have a diagnostics program called what-the-duck that checks many things about the environment.

Subsection 8.1.3 - The what-the-duck program

So, first of all, run what-the-duck. Then, fix the errors that what-the-duck shows you.

This is the proper way to run what-the-duck:

$ cd ~/duckietown
$ source environment.sh
$ git checkout master
$ git pull
$ ./dependencies_for_duckiebot.sh # if you are on a Duckiebot
$ ./dependencies_for_laptop.sh  # if you are on a laptop
$ ./what-the-duck

you have to do all the steps in the precise order.

The tool also produces an HTML report about your system which you should attach to any request for help.

How to give help

✎🔗

Creating unit tests with ROS

✎🔗
catkin_make -C catkin_ws/ --pkg easy_logs
Because of mathjax bug

Continuous integration

✎🔗

These are the conventions for the Duckietown repositories.

Road Release Process

✎🔗

You have implemented a new feature or improved an existing feature in your branch devel-project_name

A robot that is configured and able to follow lanes in Duckietown according to instructions in Unit M-24 - Checkoff: Navigation

Your branch gets merged into master and doesn’t break anything

This page is about what to do once you have developed and tested your new or improved feature and you want to contribute it back to the master branch.

If your branch is not merged into master and passes all the tests it will be lost forever.

Duckietown system

✎🔗

This part describes the Duckietown algorithms and system architecture.

We do not go in the software details. The implementation details have been already talked about at length in Part K - Software development guide.

We do give links to the ROS packages implementing the functionality.

Because of mathjax bug

Teleoperation

✎🔗

add video here

Parallel autonomy

✎🔗

to write

Because of mathjax bug

Lane control

✎🔗

video here

Indefinite navigation

✎🔗

add video here

Planning

✎🔗

add video here

Coordination

✎🔗

add video here

Duckietown ROS Guidelines

✎🔗

Fall 2017

✎🔗

Welcome to the Fall 2017 Duckietown experience.

Because of mathjax bug

The Fall 2017 Duckietown experience

✎🔗

This is the first time that a class is taught jointly across 3 continents!

There are 4 universities involved in the joint teaching for the term:

  • ETH Zürich (ETHZ), with instructors Emilio Frazzoli, Andrea Censi, Jacopo Tani.
  • University of Montreal (UdeM), with instructor Liam Paull.
  • TTI-Chicago (TTIC), with instructor Matthew Walter.
  • National Chiao Tung University (NCTU), with instructor Nick Wang.

This part of the Duckiebook describes all the information that is needed by the students of the four institutions.

At ETHZ, UdeM, TTIC, the class will be more-or-less synchronized. The materials are the same; there is some slight variation in the ordering.

Moreover, there will be some common groups for the projects.

The NCTU class is undergraduate level. Students will learn slightly simplified materials. They will not collaborate directly with the other classes.

First Steps in Duckietown

✎🔗

Onboarding Procedure

✎🔗

Welcome aboard! We are so happy you are joining us at Duckietown!

This is your onboarding procedure. Please read all the steps and then complete all the steps.

If you do not follow the steps in order, you will suffer from unnecessary confusion.

Github sign up

✎🔗

If you don’t already have a Github account, sign up now.

Github signup page

Please use your full name when it asks you. Ideally, the username should be something like FirstLast or something that resembles your name.

When you sign up, use your university email. This allows to claim an educational discount that will be useful later.

Logistics for Zürich branch

✎🔗

This section describes information specific to Zürich.

✎

These are the local TAs:

  • Shiying Li (shili@student.ethz.ch)
  • Ercan Selçuk (ercans@student.ethz.ch)
  • Miguel de la Iglesia Valls (dmiguel@student.ethz.ch)
  • Harshit Khurana (hkhurana@student.ethz.ch)
  • Dzenan Lapandic (ldzenan@student.ethz.ch)
  • Marco Erni (merni@ethz.ch)

Please contact them on Slack, rather than email.

Also feel free to contact the TAs in Montreal and Chicago.

Logistics for Montréal branch

✎🔗

This unit contains all necessary info specific for students at Univeristé de Montréal.

Logistics for Chicago branch

✎🔗

Matt

This section describes information specific to TTIC and UChicago students.

✎

The course website provides a copy of the syllabus, grading information, and details on learning objectives.

✎

Classes take place on Mondays and Wednesdays from 9am-11am in TTIC Room 530. In practice, each class will be divided into an initial lecture period (approximately one hour), followed by a lab session.

The class schedule is maintained as part of the TTIC Class Diary, which includes details on lecture topics, links to slides, etc.

✎

The following is taken from the course syllabus:

The class will assess your grasp of the material through a combination of problem sets, exams, and a final project. The contribution of each to your overall grade is as follows:

  • 20%: Problem sets
  • 10%: Checkoffs
  • 20%: Participation
  • 50%: Final project (includes report and presentation). The projects will be group-based, but we will assess the contribution of each student individually.

See the course syllabus for more information on how the participation and final project grades are determined.

✎

The following is taken from the course syllabus:

Late problem sets will be penalized 10% for each day that they are late. Those submitted more than three days beyond their due date will receive no credit.

Each student has a budget of three days that they can use to avoid late penalties. It is up to the student to decide when/how they use these days (i.e., all at once or individually). Students must identify whether and how many days they use when they submit an assignment.

It is not acceptable to use code or solutions from outside class (including those found online), unless the resources are specifically suggested as part of the problem set.

You are encouraged to collaborate through study groups and to discuss problem sets and the project in person and over Slack. However, you must acknowledge who you worked with on each problem set. You must write up and implement your own solutions and are not allowed to duplicate efforts. The correct approach is to discuss solution strategies, credit your collaborator, and write your solutions individually. Solutions that are too similar will be penalized.

✎

Duckietown labs will take place at TTIC in the robotics lab on the 4th floor.

Note: TTIC and U. Chicago students in Matthew Walter’s research group use the lab as their exclusive research and office space. It also houses several robots and hardware to support them. Please respect the space when you use it: try not to distract lab members while they are working and please don’t touch the robots, sensors, or tools.

✎

Duckietown is a collaborative effort involving close interaction among students, TAs, mentors, and faculty across several institutions. The local learning assistants (LAs) at TTIC are:

  • Andrea F. Daniele (afdaniele@ttic.edu)
  • Falcon Dai (dai@ttic.edu)
  • Jon Michaux (jmichaux@ttic.edu)
Because of mathjax bug

Logistics for NCTU branch

✎🔗

Nick and Eric (Nick’s student)

This section describes information specific to NCTU students.

✎

The Duckietown Taiwan Branch Website provides some details about Duckietown Branch in NCTU-Taiwan and results of previous class in NCTU.

✎

Classes take place on Thursday from 1:20pm~4:20om in NCTU Engineering Building 5 Room 635. Each class will be devided into two sessoins. In the first session, Professor Wang will give lessons on foundamental theory and inspire students to come up more creatitive but useful ideas on final projects. In the second sessoin, TAs will give pratical lab on how to use Duckietown platform as their project platform and use ROS as their middleware toward a fantastic work.

The class schedule is maintained as part of the NCTU Class Diary, which includes details on lecture topics, links to slides, etc.

✎

The following is taken from the course syllabus:

This course aims at developing software projects usable in real-world, and focuses on “learning by doing,” “team work,” and “research/startup oriented.”. The contribution of each to your overall grade is as follows:

  • Class Participation, In Class Quiz, Problem Sets (10%)
  • Midterm Presentation (30%)
  • Final Presentation (30%)
  • Project Report and Demo Video (30%)

See the course syllabus for more information on course object and grading policy.

✎

The following is taken from the course syllabus:

Late problem sets will be penalized 10% for each day that they are late. Those submitted more than three days beyond their due date will receive no credit.

Each student has a budget of three days that they can use to avoid late penalties. It is up to the student to decide when/how they use these days (i.e., all at once or individually). Students must identify whether and how many days they use when they submit an assignment.

It is not acceptable to use code or solutions from outside class (including those found online), unless the resources are specifically suggested as part of the problem set.

You are encouraged to collaborate through study groups and to discuss problem sets and the project in person and over Slack. However, you must acknowledge who you worked with on each problem set. You must write up and implement your own solutions and are not allowed to duplicate efforts. The correct approach is to discuss solution strategies, credit your collaborator, and write your solutions individually. Solutions that are too similar will be penalized.

✎

Duckietown labs will take place at NCTU in the same place with the lecture.

Note: The course focus on “learning by doing” which means that the lab session of each class is expectially important.

✎

Duckietown is a collaborative effort involving close interaction among students, TAs, mentors, and faculty across several institutions. The local learning assistants (LAs) at NCTU are:

  • Yu-Chieh ‘Tony’ Hsiao (tonycar12002@gmail.com)
  • Pin-Wei ‘David’ Chen (ccpwearth@gmail.com)
  • Chen-Lung ‘Eric’ Lu (eric565648@gmail.com)
  • Yung-Shan ‘Michael’ Su (michael1994822@gmail.com)
  • Chen-Hao ‘Peter’ Hung (losttime1001@gmail.com)
  • Hong-Ming ‘Peter’ Huang (peterx7803@gmail.com)
  • Tzu-Kuan ‘Brian’ Chuang (fire594594594@gmail.com)
Because of mathjax bug

Git usage guide for Fall 2017

✎🔗

Repositories

✎🔗

These are the repositories we use.

Git policy for homeworks (TTIC/UDEM)

✎🔗

This does not apply to Zurich.

Homeworks will require you to write and submit coding exercises. They will be submitted using git. Since we have a university plagiarism policy (UdeM’s, TTIC/UChicago) we have to protect students work before the deadline of the homeworks. For this reason we will follow these steps for homework submission:

  1. Go here and file a request at the bottom “Request a Discount” then enter your institution email and other info.
  2. Go to exercises-fall2017
  3. Click “Fork” button in the top right
  4. Choose your account if there are multiple options
  5. Click on the Settings tab of your repostory, not your account
  6. Under “Collaborators and Teams” in the left, click the “X” in the right for the section for “Fall 2017 Vehicle Autonomy Engineers in training”. You will get a popup asking you to confirm. Confirm.

If you have not yet cloned the duckietown repo do it now:

$ git clone git@github.com:duckietown/exercises-fall2017.git

Now you need to point the remote of your exercises-fall2017 to your new local private repo. To do, from inside your already previously cloned exercises-fall2017 repo do:

$ git remote set-url origin git@github.com:GIT_USERNAME/exercises-fall2017.git

Let’s also add an upstream remote that points back to the original duckietown repo:

$ git remote add upstream git@github.com:duckietown/exercises-fall2017.git

If you type

$ git remote -v

You should now see:

origin  git@github.com:GIT_USERNAME/exercises-fall2017.git (fetch)
origin  git@github.com:GIT_USERNAME/exercises-fall2017.git (push)
upstream  git@github.com:duckietown/exercises-fall2017.git (fetch)
upstream  git@github.com:duckietown/exercises-fall2017.git (push)

Now the next time you push (without specifying a remote) you will push to your local private repo.

Git policy for project development

✎🔗

Different than the homeworks, development for the projects will take place in the Software repo since plagiarism is not an issue here. The process is:

  1. Create a branch from master

  2. Develop code in that branch (note you may want to branch your branches. A good idea would be to have your own “master”, e.g. “your_project-master” and then do pull requests/merges into that branch as things start to work)

  3. At the end of the project submit a pull request to master to merge your code. It may or may not get merged depending on many factors.

Because of mathjax bug

Organization

✎🔗

The following roster shows the teaching staff.

Andrea Censi
Liam Paull
Jacopo Tani
First Last
Emilio Frazzoli
First Last
First Last
First Last
First Last
First Last
First Last
First Last
First Last
First Last
First Last
Greta Paull
Kirsten Bowser

Staff: To add yourself to the roster, or to change your picture, add a YAML file and a jpg file to the duckiefleet-fall2017 repository. in the people/staff directory.

The Activity Tracker

✎🔗

Link to Activity Tracker

The sheet called “Activity Tracker” describes specific tasks that you must do in a certain sequence. Tasks include things like “assemble your robot” or “sign up on Github”.

The difference between the Areas sheet and the Task sheet is that the Task sheet contains tasks that you have to do once; instead, the Areas sheet contains ongoing activities.

In this sheet, each task is a row, and each person is a column. There is one column for each person in the class, including instructors, TAs, mentors, and students.

You have two options:

  • Only use the sheet as a reference;
  • Use the sheet actively to track your progress. To do this, send a message to Kirsten with your gmail address, and add yourself.

Each task in the first column is linked to the documentation that describes how to perform the task.

The colored boxes have the following meaning:

  • Grey: not ready. This means the task is not ready for you to start yet.
  • Red: not started. The person has not started the task.
  • Blue: in progress. The person is doing the task.
  • Yellow: blocked. The person is blocked.
  • Green: done. The person is done with the task.
  • n/a: the task is not applicable to the person. (Certain tasks are staff-only.)

The Areas sheet

✎🔗

Please familiarize yourself with this spreadsheet and bookmark it in your browser.

The sheet called “Areas” describes the points of contact for each part of this experience. These are the people that can offer support. In particular, note that we list two points of contact: one for America, and one for Europe. Moreover, there is a link to a Slack channel, which is the place where to ask for help. (We’ll get you started on Slack in just a minute.)

Because of mathjax bug

Getting and giving help

✎🔗

Slack Channels

✎🔗

This page describes all of the helpful Slack channels and their purposes so that you can figure out where to get help.

Channels

✎🔗

You can also easily join the ones that you are interested in by clicking the links in this message.

Duckietown Slack Channels
Channel Purpose
help-accounts Info about necessary accounts, such as Slack, Github, etc.
help-assembly Help putting your robot together
help-camera-calib Help doing the intrinsic and extrinsic calibration of your camera
help-duckuments Help compiling the online documentation
help-git Help with git
help-infrastructure Help with software infrastructure, such as Makefiles, unit tests, continuous integration, etc.
help-laptops Help getting your laptop setup with Ubuntu 16.04
help-parts Help getting the parts for the robot or replacement parts if you broke something
help-robot-setup Help getting the robot setup to do basic things like be driven with a joystick
help-ros Help with the Robot Operating System (ROS)
help-wheel-calib Help doing your odometry calibration
comment

Note that we can link directly to the channels. (See list in the org sheet.) -AC

Because of mathjax bug

Zürich branch diary

✎🔗

Wed Sep 20: Welcome to Duckietown!

✎🔗

This was an introduction meeting.

Monday Sep 25: Introduction to autonomy

✎🔗

Monday Sep 25, late at night: Onboarding instructions

✎🔗

At some late hour of the night, we sent out the onboarding instructions.

Section 2.1 - Onboarding Procedure

Please complete the onboarding questionnaire by Tuesday, September 26, 15:00.

Wednesday Sep 27: Duckiebox distribution, and getting to know each other

✎🔗

Today we distribute the Duckieboxes and we name the robots. In other words, we perform the Duckiebox ceremony.

  • getting to know each other;
  • naming the robots;
  • distribute the Duckieboxes.

If you cannot make it to this class for the Duckiebox distribution, please inform the TA, to schedule a different time.

Preparation, step 1: choose a name for your robot

✎🔗

Before arriving to class, you must think of a name for your robot.

Here are the constraints:

  • The name must work as a hostname. It needs to start with a letter, contains only letters and numbers, and no spaces or punctuation.
  • It should be short, easy to type. (You’ll type it a lot.)
  • It cannot be your own name.
  • It cannot be a generic name like “robot”, “duckiebot”, “car”. It cannot contain brand names.

Preparation, step 2: prepare a brief elevator pitch

✎🔗

As members of the same community, it is important to get to know a little about each other, so to know who to rely on in times of need.

During the Duckiebox distribution ceremony, you will be asked to walk up to the board, write your name on it, and introduce yourself. Keep it very brief (30 seconds), and tell us:

  • what is your professional background and expertise / favorite subject;
  • what is the name of your robot;
  • why did you choose to name your robot in that way.

You will then receive a Duckiebox from our senior staff, a simple gesture but of sempiternal glory, for which you have now become a member of the Duckietown community. This important moment will be remembered through a photograph. (If in the future you become a famous roboticist, we want to claim it’s all our merit.)

Finally, you will bring the Duckiebox to our junior staff, who will apply labels with your name and the name of the robot. They will also give you labels with your robot name for future application on your Duckiebot.

Thursday Sep 28: Misc announcements

✎🔗

We created the channel #ethz-chitchat for questions and other random things, so that we can leave this channel #ethz-announcements only for announcements.

We sent the final list to the Department; so hopefully in a couple of days the situation on MyStudies is regularized.

The “lab” time on Friday consists in an empty room for you to use as you wish, for example to assembe the robots together. In particular, it’s on the same floor of the Duckietown room and the IDSC lab.

The instructions for assembling the Duckiebots are here. Note that you don’t have to do the parts that we did for you: buying the parts, soldering the boards, and reproducing the image.

Expected progress: We are happy if we see everybody reaching up to Unit C-14 - RC+camera remotely by Monday October 9. You are encouraged to start very early; it’s likely that you will not receive much help on Sunday October 8…

Because of mathjax bug

Sep 28: some announcements

✎🔗

A couple of announcements:

  1. We created #ethz-chitchat for questions and other random things, so that we can leave this channel #ethz-announcements only for announcements.

  2. MyStudies should be updated with everybody’s names.

  3. The “lab” time tomorrow consists in an empty room for you to use as you wish, for example to assemble the robots together. In particular, it’s on the same floor of the Duckietown room and the IDSC lab.

  4. The instructions for assembling the Duckiebots are here. Note that we did for you step I-2 (buying the parts) and I-3 (soldering the boards); and I-6 is optional.

  5. We are happy if we see everybody reaching I-13 by the Monday after next. I encourage you to start sooner than later.

  6. I see only 30 people in this channel instead of 42. Please tell your friends that now all the action is on Slack.

Oct 01 (Mon): Announcement

✎🔗

It looks like that the current documentation is misleading in a couple of points. This is partially due to the fact that there is some divergence between Chicago, Montreal and Zurich regarding (1) the parts given out and (2) the setup environment (which networks are available). We did the simple changes (e.g. removing the infamous part 6), but we need some more time to review the other issues. At this point, the best course of action is that you enjoy your weekend without working on Duckietown, while we spend the weekend fixing the documentation.

Oct 02 (Mon): Networking, logical/physical architectures

✎🔗

Oct 04 (Wed): Modeling

✎🔗

Oct 09 (Mon): Autonomy architectures and version control

✎🔗

Oct 11 (Wed): Computer vision and odometry calibration

✎🔗

Oct 13 (Fri): new series of tasks out

✎🔗

Taking a video of the joystick control

✎🔗

Please take a video of the robot as it drives with joystick control, as described in Section 17.7 - Upload your video and upload it according to the instructions.

Example of a great video, but with a nonconforming Duckiebot.

Camera calibration

✎🔗

Go forth and calibrate the camera! And get help in #help-camera-calib.

Wheel calibration

✎🔗

This is not ready yet! will be ready in a day or so.

Taking a log check off

✎🔗

Follow the instructions here to learn how to take a log.

Data processing exercises

✎🔗

See the list of exercises here.

Get help in #ex-data-processing.

Current deadlines for Zurich
Robot assembly overdue
Robot/laptop configuration overdue
Subsection 11.13.1 - Taking a video of the joystick control Monday Oct 16
Subsection 11.13.2 - Camera calibration Friday Oct 20
Subsection 11.13.3 - Wheel calibration not ready yet
Subsection 11.13.4 - Taking a log check off Wed Oct 18
Subsection 11.13.5 - Data processing exercises Monday Oct 23

Oct 16 (Mon): Line detection

✎🔗

Oct 18 (Wed): Feature extraction

✎🔗

Oct 20 (Fri): Lab session

✎🔗

Oct 23 (Mon) Filtering I

✎🔗

Oct 25 (Wed) Filtering II

✎🔗

a - Lectures (Particle Filter) PowerPoint presentation, PDF.

b - Lectures (Lane Filter) PowerPoint presentation, PDF.

Nov 1 (Wed) Control Systems

✎🔗

a - Lectures (Control Systems Module I) PowerPoint presentation, PDF.

b - Lectures (Control Systems Module II) PowerPoint presentation, PDF.

Points to be noted - Running what-the-duck on laptop and Duckiebot is mandatory. It helps save time in debugging errors and also is a standard way to ask for help from the staff. Keep repeating it periodically so as to keep the data up-to date - For the people lacking calibrated wheels, this serves as a reminder to calibrate the wheels and keep their duckiebot up-to date - It is advised to fill the lecture feedback form (Feedback form), so as to increase the effectiveness of the lectures - Always check the consistency of the camera calibration checkerboard before camera calibration (one has to check for the correct square size and correct distance between world and checkerboard reference)

Nov 6 (Mon) Project Pitches

✎🔗

Lecture Project Pitches PDF.

Nov 8 (Wed) Motion Planing

✎🔗

Lecture Motion Planing PDF.

A few references for planning of Andrea Censi:

Nov 13 (Mon) Project Team Assignments

✎🔗
  • First Lecture: Project Team Assignments PDF.
  • Second Lecture: First meeting of the Controllers group –> Filling out the Preliminary Design Document

Nov 15 (Wed) Putting things together

✎🔗
  • First Lecture: Putting things together PDF.
  • Second Lecture: Second meeting of the Controllers group –> Filling out the Preliminary Design Document

Nov 22 (Wed) Fleet Control

✎🔗
  • Lecture: Fleet Control in Autonomous Mobility on Demand PDF.

Nov 27 (Mon) Inermediate design Report

✎🔗

  • First Lecture: The intermediate Design report is introduced heretemplate-int-report. It is due on Monday 4th of December.

  • Second Lecture was left for project discussion and interaction.

Nov 29 (Wed) Fleet Control

✎🔗

  • First Lecture: Clausdio finished Fleet Control in Autonomous Mobility on Demand PDF.

  • Second Lecture Julian Presented the state of the art in data driven vs Model driven robotics. PDF

Because of mathjax bug

Montréal branch diary

✎🔗

Oct 30 (Mon)

✎🔗

XXX

Nov 01 (Wed)

✎🔗

XXX

Nov 06 (Mon)

✎🔗

XXX

Nov 08 (Wed)

✎🔗

XXX

Nov 13 (Mon)

✎🔗

XXX

Nov 15 (Wed)

✎🔗

XXX

Nov 20 (Mon)

✎🔗

XXX

Nov 22 (Wed)

✎🔗

XXX

Nov 27 (Mon)

✎🔗

XXX

Nov 29 (Wed)

✎🔗

XXX

Dec 04 (Mon)

✎🔗

XXX

Dec 06 (Wed)

✎🔗

XXX

Dec 11 (Mon)

✎🔗

XXX

Chicago branch diary

✎🔗

Classes take place on Mondays and Wednesdays from 9am-11am in TTIC Room 530.

NCTU branch diary

✎🔗

Classes take place on Thursday from 1:20pm-4:20pm in NCTU Engineering Building 5 Room 635.

Slack Channels

✎🔗

This page describes all of the helpful Slack channels and their purposes so that you can figure out where to get help.

Guide for TAs

✎🔗

Checkoff: Assembly and Configuration

✎🔗

The first job is to get your Duckiebot put together and up and running.

Upload your video

✎🔗

You should record a video demonstrating that your Duckiebot is up and running. Brownie points for creative videos. Please upload your videos via the following URL:

Chicago: upload your video

Zurich: upload your video

Because of mathjax bug

Checkoff: Take a Log

✎🔗

Unit M-17 - Checkoff: Assembly and Configuration

A verified log in rosbag format uploaded to Dropbox.

#help-logging

Montreal deadline: Oct 4, 11pm

Zurich deadline: Oct 20, 17:00

Checkoff: Robot Calibration

✎🔗

Unit M-17 - Checkoff: Assembly and Configuration

That you have correctly cloned and followed the git procedure outline in Unit M-7 - Git usage guide for Fall 2017

That you have correctly setup your environment variables according to Section 4.1 - Environment variables (updated Sept 12)

You robot calibrations (wheels and camera (x2)) are merged to git through a PR

Slack channels: #help-wheel-calib, #help-camera-calib

Homework: Data Processing (UdeM)

✎🔗

Unit M-18 - Checkoff: Take a Log

Unit J-30 - ROS installation and reference

Ability to perform basic operations on images

Build your first ROS package and node

Ability to process imagery live

#ex-data_processing

Montreal deadline: Oct 4, 11:00pm

Homework: Data Processing (TTIC)

✎🔗

Unit J-30 - ROS installation and reference

Ability to perform basic operations on images

Build your first ROS package and node

Ability to process imagery live

#help-data-processing

TTIC deadline: Friday, October 13 11:59pm CT

Exercises: Data Processing (Zurich)

✎🔗

Unit J-30 - ROS installation and reference

Ability to perform basic operations on images

Build your first ROS package and node

Ability to process imagery live

Slack channel to get help: #ex-data-processing

Homework: Augmented Reality

✎🔗

Unit J-30 - ROS installation and reference

Unit C-17 - Camera calibration

Ability to project fake things from an image back into the world

#ex-augmented_reality

Montreal deadline: Oct 27, 11:00pm
Chicago deadline: Oct 27, 11:59pm
Zurich deadline: Oct ???, 11:59pm

Checkoff: Navigation

✎🔗

Unit M-19 - Checkoff: Robot Calibration

Unit M-18 - Checkoff: Take a Log

2 logs of your robot autonomously navigating Duckietown

Montreal Deadline: Nov 15, 11pm

Chicago Deadline: Nov 15, 11pm

#help-navigation

Homework: Lane Filtering

✎🔗
Montreal deadline: Nov 17, 11:00pm
Chicago deadline: Nov 17, 11:59pm

#ex-filtering

Guide for mentors

✎🔗

Liam?

Because of mathjax bug

Project proposals

✎🔗

to write

Because of mathjax bug

System Architecture

✎🔗

Mission 1

✎🔗

Ensure that the development and integration of the projects into the system goes smoothly and that the resulting system makes sense, and is useful for future duckierations (duckie + generations).

Mission 2

✎🔗

Where there is a system, there is a want (nay, need) for optimisation. Describing a system’s performance and resource requirements in a quantifiable way is a critical part of being able to benchmark modules and optimise the system.

Mission 2 is to formalise the description of the system characteristics, so that eventually the system performance can be optimised for some given resources.

Template of a project

✎🔗

Make a copy of this document before editing.

The Map Description

✎🔗

The map to be used in the Fall 2017 class is shown in Figure 30.1.

The map to be used in the Fall 2017 class

The editable keynote file is in this directory of the duckuments repo. The ids on the signs correspond to the Apriltag IDs. For more details see Unit D-3 - Signage.

Because of mathjax bug

Fall 2017 projects

✎🔗

Welcome to the Fall 2017 projects.

Instructions for using the template

✎🔗

  1. Make a copy of the template 10_templates folder and paste it inside /atoms_85_fall2017_projects.

  2. Rename the folder to the next available integer followed by the short group name. E.g.: 10_templates becomes 11_first_group_name for the first group, then 12_second_group_name for the second, and so forth.

  3. Edit the 10-preliminary-design-document-template file name by substituting template with group-name

  4. Open the preliminary design document and personalize the template to your group.

All groups have got their unique ID number and folders are renamed according to the following table. You are allowed and encouraged to use short names. Please merge from the master. New pull requests conflicting to this table will be rejected.

Group name: preliminary design document

✎🔗

Group name: intermediate report

✎🔗

It’s time to commit on what you are building, and to make sure that it fits with everything else.

This consists of 3 parts:

  • Part 1: System interfaces: Does your piece fit with everything else? You will have to convince both system architect and software architect and they must sign-off on this.

  • Part 2: Demo and evaluation plan: Do you have a credible plan for evaluating what you are building? You will have to convince the VPs of Safety and they must sign-off on this.

  • Part 3: Data collection, annotation, and analysis: Do you have a credible plan for collecting, annotating and analyzing the data? You will have to convince the data czars and they must sign-off on this.

Intermediate Report Supervisors
System Architects Sonja Brits, Andrea Censi
Software Architects Breandan Considine, Liam Paull
Vice President of Safety Miguel de la Iglesia, Jacopo Tani
Data Czars Manfred Diaz, Jonathan Aresenault

The Heroes quests

✎🔗

The “Heroes” team is a special task force with the responsibility to make sure that “everything works” and create a smooth experience for the rest of the teams, in terms of developing own projects, integration with other teams and documentation. Apart from that, each of the heroes will also have their own individual quest…

PDD - Smart City

✎🔗

PDD - System Identification

✎🔗

System Identification: Intermediate Report

✎🔗

PDD - The Controllers

✎🔗

The Controllers: Intermediate Report

✎🔗

Intermediate Report Supervisors
System Architects Sonja Brits, Andrea Censi
Software Architects Breandan Considine, Liam Paull
Vice President of Safety Miguel de la Iglesia, Jacopo Tani
Data Czars Manfred Diaz, Jonathan Aresenault

Conventions used in the following document
Variable Description
$d_{ref}$ Reference distance from center of lane
$d_{act}$ Actual distance from center of lane
$d_{est}$ Estimated distance from center of lane
$\theta_{act}$ Actual angle between robot and center of lane
$\theta_{est}$ Estimated angle between robot and center of lane
$c_{act}$ Actual curvature of lane
$c_{est}$ Estimated curvature of lane
$c_{ref}$ Reference curvature of the path to follow
$v_{ref}$ Reference velocity

image

PDD - Saviors

✎🔗

The Saviors: intermediate report

✎🔗

PDD - Parking

✎🔗

Parking: intermediate report

✎🔗

PDD - Explicit Coordination

✎🔗

Explicit Coordination: Intermediate Report

✎🔗

PDD - Implicit Coordination

✎🔗

Implicit Coordination: intermediate report

✎🔗

PDD - Distributed Estimation

✎🔗

PDD - Fleet-level Planning

✎🔗

Fleet Level Planning: Intermediate Report

✎🔗

This document describes system that the Fleet-planning team is planning to implement. Part 1 describes the interfaces to other parts of the system. I.e. How it communicates with all of them. In Part 2 a plan for the demo and evaluation is presented. And finally part 3 focuses on data collection, annotation and analysis. As for this project not much annotated data is used, part 3 is rather short.

Transferred Lane following

✎🔗

This is the description of transferred lane following demo.

Wheels calibration completed.wheel calibration

Camera installed in the right position.

Joystick demo has been successfully launched.Joystick demo

PyTorch installed on duckiebot and laptop. (On duckiebot, you can either build from source(link), or download the pytorch-0.2 wheel file we built)

Video of expected results

✎🔗

link 1 of lane following link 2 of lane following

Duckietown setup notes

✎🔗

A duckietown with white and yellow lanes. No obstacles on the lane.

Duckiebot setup notes

✎🔗

Make sure the camera is heading ahead. Tighten the screws if necessary.

Pre-flight checklist

✎🔗

Check: turn on joystick.

Check: Enough battery of the duckiebot.

Demo instructions

✎🔗

Here, give step by step instructions to reproduce the demo.

Step 1: On duckiebot, in /DUCKIERTOWN_ROOT/ directory, run command:

duckiebot `roslaunch deep_lane_following lane_following.launch`

Wait a while so that everything has been launched. Press R1 to start autonomous lane following. Press L1 to switch to joystick control.

The following is the same as demo-lane-following: Empirically speaking, no duckiebot will successfully run the demo for its first time. Parameter tuning is a must. The only two parameters that you can modify is the gain and trim. The parameter pair which makes your bot go straight will unlikely work for the lane following due to the current controller design. Facts show that a gain ranging from 0.5 to 0.9, as long as paired with a suitable trim, will all work on this demo. Start with your parameter pair obtained from wheel calibration. Increase gain for higher speed. Increase trim to horizontally move the bot to the center of the lane. Decrease will do the inverse.

Step 2: On laptop, make sure ros environment has been activated, run command:

laptop `rqt`

In rqt, the images can be visualized are /(vehicle_name)/camera_node/image/compressed

Troubleshooting

✎🔗

Contact Chip Schaff or Ruotian Luo(TTIC) via Slack if any trouble occurs.

Because of mathjax bug

Transfer Learning in Robotics

✎🔗

Understanding transfer learning and the domain randomization technique for transfer learning.

This unit introduces the concept of Transfer Learning and how it can be applied to Robotics.

Transfer Learning Definition

✎🔗

Transfer learning is a subfield of machine learning that focuses on using knowledge gained while solving one problem to solve a related problem.

PDD - Supervised-learning

✎🔗

Supervised Learning: intermediate report

✎🔗

PDD Neural Slam

✎🔗

PDD - Visual Odometry

✎🔗

Visual Odometry Project

✎🔗

Here we briefly describe the theory behind the model in the visual odometry project. The discussion begins with a review of epipolar geometry and a description of the depth image-based rendering problem, then moves to the description of the deep learning model used.

Deep Visual Odometry ROS Package

✎🔗

PDD - Anti-Instagram

✎🔗

Transfer: preliminary design document

✎🔗

Packages - Infrastructure

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package duckieteam

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

Code for handling the DB of people and robots.

create-machines

✎🔗

This program creates the machines file, using the data contained in the scuderia database.

Run as follows:

$ rosrun duckieteam create-machines

create-roster

✎🔗

this program is unfinished

Because of mathjax bug
Because of mathjax bug

Package duckietown

✎🔗

✎

Link to package on Github

✎

Mack

✎

The duckietown meta package

Because of mathjax bug
Because of mathjax bug

Package duckietown_msgs

✎🔗

✎

Link to package on Github

✎

Mack

✎

Add a description of package duckietown_msgs in package.xml.

Because of mathjax bug
Because of mathjax bug

Package easy_algo

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

A package to make it easy to abstract about algorithm configuration and create regression tests.

Package easy_logs

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

The easy_logs packages manages the Duckietown logs.

It indexes them and has a simple interface to query the DB.

Tests should use the easy_logs DB to know what logs to work on.

Selector language

✎🔗

Here are some examples for the query language (Table 5.1).

Show all the Ferrari logs:

$ rosrun easy_logs summary vehicle:ferrari

All the logs of length less than 45 s:

$ rosrun easy_logs summary "length:<45"

All the invalid logs:

$ rosrun easy_logs summary "length:<45,valid:False"

All the invalid logs of length less than 45 s:

$ rosrun easy_logs summary "length:<45,valid:False"
Query language
expression example explanation
attribute:expr vehicle:ferrari Checks that the attribute [attribute of the object satisfies the expression in expr
>lower bound >10 Lower bound
<upper bound <1 Upper bound
expr1,expr2 >10,<20 And between two expressions
expr1+expr2 <5+>10 Or between two expressions
pattern *ferrari* Other strings are interpreted as wildcard patterns.

Automatic log download using download

✎🔗

The command-line program download downloads requires logs from the cloud.

The syntax is:

$ rosrun easy_logs download log name

For example:

$ rosrun easy_logs download 20160223-amadoa-amadobot-RCDP2

If the file 20160223-amadoa-amadobot-RCDP2.bag is not available locally, it is downloaded.

The database of URLs is at the file dropbox.urls.yaml in the package easy_node.

A typical use case would be the following, in which a script needs a log with which to work.

Using the download program we declare that we need the log. Then, we use find to find the path.

#!/bin/bash
set -ex

# We need the log to proceed
rosrun easy_logs download 20160223-amadoa-amadobot-RCDP2

# Here, we know that we have the log. We use `find` to get the filename.
filename=`rosrun easy_logs find 20160223-amadoa-amadobot-RCDP2`

# We can now use the log
vdir ${filename}

Browsing the cloud

✎🔗

How do you know which logs are in the cloud?

Run the following to download a database of all the logs available in the cloud:

$ make cloud-download

You can query the DB by using summary with the option --cloud:

$ rosrun easy_logs summary --cloud '*RCDP6*'

| #     Log name                                   date          length    vehicle name
| --    ---------------------------------------    ----------    ------    ------------
| 0     20160122-censi-ferrari-RCDP6-catliu        2016-02-28     194 s    ferrari
| 1     20160122-censi-ferrari-RCDP6-joe-wl        2016-02-27     196 s    ferrari
| 2     20160228-sanguk-setlist-RCDP6-sangukbo     2016-03-02     193 s    ferrari
| 3     20160122-censi-ferrari-RCDP6-eharbitz      2016-02-27     198 s    ferrari
| 4     20160122-censi-ferrari-RCDP6-teddy         2016-02-28     193 s    ferrari
| 5     20160122-censi-ferrari-RCDP6-jenshen       2016-02-29      83 s    ferrari

Then, you can download locally using:

$ rosrun easy_logs download 20160122-censi-ferrari-RCDP6-teddy

Once it is downloaded, the log becomes part of the local database.

Advanged log indexing and generation

✎🔗

Shuffle

✎🔗

expr/shuffle shuffles the order of the logs in expr.

Give me all the oreo logs, in random order:

$ rosrun easy_logs summary  vehicle:oreo/shuffle

Simple indexing

✎🔗

expr/[i] takes the i-th entry.

Give me the first log:

$ rosrun easy_logs summary  "vehicle:oreo/[0]"

Give me a random log; i.e. the first of a random list.

$ rosrun easy_logs summary  "vehicle:oreo/shuffle/[0]"

Complex indexing

✎🔗

You can use the exact Python syntax for indexing, including [a:], [:b], [a:b], [a:b:c], etc.

Give me three random logs:

$ rosrun easy_logs summary  "all/shuffle/[:3]"

Sorting

✎🔗

To implement.

Time indexing

✎🔗

You can ask for only a part of a log using the syntax:

expr/{start:stop}
expr/{start:}
expr/{:stop}

where start and stop are in time relative to the start of the log.

For example, “give me all the first 1-second intervals of the logs” is

all/{:1}

Cut the first 3 seconds of all the logs:

all/{3:}

Give me the interval between 30 s and 35 s:

all/{30:35}

How to set up the backend needed to use files from the cloud

✎🔗

This applies to any resource, not only logs.

First, put the file in the duckiedown-data-2017 directory on Dropbox.

This is the link.

You need to ask Liam/Andrea because only them have write access.

Then, get the public link address and put it in the file dropbox.urls.yaml in the package easy_node. Remember to have ?dl=1 instead of ?dl=0 in the url.

In the code, use the function require_resource():

from duckietown_utils import require_resource

zipname = require_resource('ii-datasets.zip')

The storage area is in ${DUCKIETOWN_ROOT}/cache/download.

If the file is already downloaded, it is not downloaded again.

(So, if the file changes, you need to delete the cache directory. The best practice is to change the filename every time the file changes.)

The function require_resource() returns the path to the downloaded file.

Also note that you can put any URL in the file dropbox.urls.yaml; but the convention is that we only link things to Dropbox.

Because of mathjax bug
Because of mathjax bug

Package easy_node

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

easy_node is a framework to make it easier to create and document ROS nodes.

The main idea is to provide a declarative approach to describe:

  • The node parameters;
  • The node’s subscriptions;
  • The node’s publishers;
  • The node’s assumptions (contracts).

The user describes subscriptions, publishers, and parameters in a YAML file.

The framework then automatically takes care of:

  • Calling the necessary boilerplate ROS commands for subscribing and publishing topics.
  • Loading and monitoring configuration.
  • Create the Markdown documentation that describes the nodes.
  • Provide a set of common functionality, such as benchmarking and monitoring latencies.

Using easy_node allows to cut 40%-50% of the code required for programming a node. For an example, see the package line_detector2, which contains a re-implementation of line_detector using the new framework.

Transition plan: The plan is to first use easy_node just for documenting the nodes. Then, later, convert all the nodes to use it.

Benchmarking

✎🔗

EasyNode implements some simple timing statistics. These are accessed using the context object passed to the message received callbacks.

Here’s an example use, from line_detector2:

def on_received_image(self, context, image_msg):

    with context.phase('decoding'):
        ...

    with context.phase('resizing'):
        # Resize and crop image
        ...

    stats = context.get_stats()
    self.info(stats)

The idea is to enclose the different phases of the computation using the context manager phase(name).

A summary of the statistics can be accessed by using context.get_stats().

For example, this will print:

Last 24.4 s: received 734 (30.0 fps) processed 301 (12.3 fps) skipped 433 (17.7 fps) (59 %)
            decoding | total latency  25.5 ms | delta wall   20.7 ms | delta clock  20.7 ms
            resizing | total latency  26.6 ms | delta wall    0.8 ms | delta clock   0.7 ms
          correcting | total latency  29.1 ms | delta wall    2.2 ms | delta clock   2.2 ms
           detection | total latency  47.7 ms | delta wall   18.2 ms | delta clock  21.3 ms
    preparing-images | total latency  55.0 ms | delta wall    7.0 ms | delta clock   7.0 ms
          publishing | total latency  55.5 ms | delta wall    0.1 ms | delta clock   0.1 ms
          draw-lines | total latency  59.7 ms | delta wall    4.0 ms | delta clock   3.9 ms
    published-images | total latency  61.2 ms | delta wall    0.9 ms | delta clock   0.8 ms
pub_edge/pub_segment | total latency  86.3 ms | delta wall   24.7 ms | delta clock  24.0 ms

Package easy_regression

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

A package to make it easy to abstract about algorithm configuration and create regression tests.

Package what_the_duck

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

what-the-duck is a program that tests dozens of configuration inconsistencies that can happen on a Duckiebot.

The what-the-duck program

✎🔗

The proper usage of what-the-duck to debug an environment problem is the following sequence:

$ # open a new terminal
$ cd Duckietown root
$ git checkout master
$ git pull
$ source environment.sh
$ ./dependencies_for_duckiebot.sh # if you are on a Duckiebot
$ ./dependencies_for_laptop.sh    # if you are on a laptop
$ make build-clean
$ make build-catkin
$ ./what-the-duck

you have to do all the steps in the precise order.

✎

The telemetry is collected and available at this URL.

Because of mathjax bug

Packages - Teleoperation

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package adafruit_drivers

✎🔗

✎

Link to package on Github

✎

Dmitry Yershov

Mack

✎

Add a description of package adafruit_drivers in package.xml.

These are the Adafruit drivers.

What is the original location of this package?

Because of mathjax bug
Because of mathjax bug

Package dagu_car

✎🔗

✎

Link to package on Github

✎

Dmitry Yershov

Shih-Yuan Liu

Mack

✎

Add a description of package dagu_car in package.xml.

Node forward_kinematics_node

✎🔗

(Generated from configuration forward_kinematics_node.easy_node.yaml.)

Missing node description in forward_kinematics_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node inverse_kinematics_node

✎🔗

(Generated from configuration inverse_kinematics_node.easy_node.yaml.)

Missing node description in inverse_kinematics_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node velocity_to_pose_node

✎🔗

(Generated from configuration velocity_to_pose_node.easy_node.yaml.)

Missing node description in velocity_to_pose_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node car_cmd_switch_node

✎🔗

(Generated from configuration car_cmd_switch_node.easy_node.yaml.)

Missing node description in car_cmd_switch_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node wheels_driver_node

✎🔗

(Generated from configuration wheels_driver_node.easy_node.yaml.)

Missing node description in wheels_driver_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node wheels_trimmer_node

✎🔗

(Generated from configuration wheels_trimmer_node.easy_node.yaml.)

Missing node description in wheels_trimmer_node.easy_node.yaml.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Because of mathjax bug
Because of mathjax bug

Package joy_mapper

✎🔗

✎

Link to package on Github

✎

Mack

✎

The joy_mapper package for duckietown. Takes sensor_msgs.Joy and convert it to duckietown_msgs.CarControl.

Package pi_camera

✎🔗

✎

Link to package on Github

✎

Andrea Censi

✎

Add a description of package pi_camera in package.xml.

Node camera_node_sequence

✎🔗

(Generated from configuration camera_node_sequence.easy_node.yaml.)

Camera driver, second approach.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node camera_node_continuous

✎🔗

(Generated from configuration camera_node_continuous.easy_node.yaml.)

Camera driver.

✎

Parameter framerate: float; default value: 60.0

Frame rate

Parameter res_w: int; default value: 320

Resolution (width)

Parameter res_h: int; default value: 200

Resolution (height)

✎

No subscriptions defined.

✎

Publisher image_compressed: topic ~image/compressed" (CompressedImage)

Missing description for entry “image_compressed”.

Node decoder_node

✎🔗

(Generated from configuration decoder_node.easy_node.yaml.)

A node that decodes a compressed image into a regular image. This is useful so that multiple nodes that need to use the image do not do redundant computaon.

✎

Parameter publish_freq: float; default value: 1.0

Frequency at which to publish (Hz).

✎

Subscription compressed_image: topic ~compressed_image (CompressedImage)

The image to decode.

Subscription switch: topic ~switch (BoolStamped)

Switch to turn on or off. The node starts as active.

✎

Publisher raw: topic ~image/raw (Image)

The decoded image.

Node img_process_node

✎🔗

(Generated from configuration img_process_node.easy_node.yaml.)

Apparently, a template, or a node never finished.

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Node cam_info_reader_node

✎🔗

(Generated from configuration cam_info_reader_node.easy_node.yaml.)

Publishes a CameraInfo message every time it receives an image.

✎

Parameter config: str; default value: 'baseline'

Missing description for entry “config”.

Parameter cali_file_name: str; default value: 'default'

Missing description for entry “cali_file_name”.

Parameter image_type: str; default value: 'compressed'

Missing description for entry “image_type”.

✎

Subscription compressed_image: topic ~compressed_image (CompressedImage)

If image_type is “compressed” then it’s CompressedImage, otherwise Image.

✎

Publisher camera_info: topic ~camera_info (CameraInfo)

Missing description for entry “camera_info”.

Because of mathjax bug

Packages - Lane control

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package anti_instagram

✎🔗

✎

Link to package on Github

✎

mfe

Mack

✎

Add a description of package anti_instagram in package.xml.

Node anti_instagram_node

✎🔗

(Generated from configuration anti_instagram_node.easy_node.yaml.)

Missing node description in anti_instagram_node.easy_node.yaml.

✎

Parameter publish_corrected_image: bool; default value: False

Whether to compute and publish the corrected image.

✎

Subscription image: topic ~uncorrected_image (CompressedImage)

This is the compressed image to read.

Subscription click: topic ~click (BoolStamped)

Activate the calibration phase with this switch.

✎

Publisher image: topic ~corrected_image (Image)

The corrected image.

Publisher health: topic ~colorSegment (AntiInstagramHealth)

The health of the process.

Publisher transform: topic ~transform (AntiInstagramTransform)

The computed transform.

Because of mathjax bug
Because of mathjax bug

Package complete_image_pipeline

✎🔗

✎

Link to package on Github

✎

Mack

✎

This package contains functions that allow to call the entire message pipeline programmatically, as a Python function, without having to instantiate any ROS node.

This is useful for regression tests, and quick tests with new logs.

Package ground_projection

✎🔗

✎

Link to package on Github

✎

Changhyun Choi

Mack

✎

Add a description of package ground_projection in package.xml.

Node ground_projection_node

✎🔗

(Generated from configuration ground_projection.easy_node.yaml.)

node description for ground_projection

✎

No parameters defined.

✎

No subscriptions defined.

✎

No publishers defined.

Because of mathjax bug
Because of mathjax bug

Package lane_control

✎🔗

✎

Link to package on Github

✎

steven

Shih-Yuan Liu

Liam Paull

✎

Add a description of package lane_control in package.xml.

lane_controller_node

✎🔗

(Generated from configuration lane_controller_node.easy_node.yaml.)

there is some very funny business inside. It appears that k_d and k_theta are switched around.

✎

Parameter v_bar: float

Nominal linear velocity (m/s).

Parameter k_theta: float

Proportional gain for $\theta$.

Parameter k_d: float

Propertional gain for $d$.

Parameter d_thres: float

Cap for error in $d$.

Parameter theta_thres: float

Maximum desired $\theta$.

Parameter d_offset: float

A configurable offset from the lane position.

✎

Subscription lane_reading: topic ~lane_pose (LanePose)

Missing description for entry “lane_reading”.

✎

Publisher car_cmd: topic ~car_cmd (Twist2DStamped)

Missing description for entry “car_cmd”.

Because of mathjax bug
Because of mathjax bug

Package lane_filter

✎🔗

Liam

✎

Link to package on Github

✎

Liam Paull

✎

Add a description of package lane_filter in package.xml.

lane_filter_node

✎🔗

(Generated from configuration lane_filter_node.easy_node.yaml.)

Missing node description in lane_filter_node.easy_node.yaml.

✎

Parameter mean_d_0: float; default value: 0.0

Missing description for entry “mean_d_0”.

Parameter mean_phi_0: float; default value: 0.0

Missing description for entry “mean_phi_0”.

Parameter sigma_d_0: float; default value: 0.0

Missing description for entry “sigma_d_0”.

Parameter sigma_phi_0: float; default value: 0.0

Missing description for entry “sigma_phi_0”.

Parameter delta_d: float; default value: 0.02

(meters)

Parameter delta_phi: float; default value: 0.0

(radians)

Parameter d_max: float; default value: 0.5

Missing description for entry “d_max”.

Parameter d_min: float; default value: -0.7

Missing description for entry “d_min”.

Parameter phi_min: float; default value: -1.5707

Missing description for entry “phi_min”.

Parameter phi_max: float; default value: 1.5707

Missing description for entry “phi_max”.

Parameter cov_v: float; default value: 0.5

Linear velocity “input”.

XXX which units?

Parameter cov_omega: float; default value: 0.01

Angular velocity “input”.

XXX which units?

Parameter linewidth_white: float; default value: 0.04

Missing description for entry “linewidth_white”.

Parameter linewidth_yellow: float; default value: 0.02

Missing description for entry “linewidth_yellow”.

Parameter lanewidth: float; default value: 0.4

Missing description for entry “lanewidth”.

Parameter min_max: float; default value: 0.3

Expressed in nats.

Parameter use_distance_weighting: bool; default value: False

For use of distance weighting (dw) function.

Parameter zero_val: float; default value: 1.0

Missing description for entry “zero_val”.

Parameter l_peak: float; default value: 1.0

Missing description for entry “l_peak”.

Parameter peak_val: float; default value: 10.0

Missing description for entry “peak_val”.

Parameter l_max: float; default value: 2.0

Missing description for entry “l_max”.

Parameter use_max_segment_dist: bool; default value: False

For use of maximum segment distance.

Parameter max_segment_dist: float; default value: 1.0

For use of maximum segment distance.

Parameter use_min_segs: bool; default value: False

For use of minimum segment count.

Parameter min_segs: int; default value: 10

For use of minimum segment count.

Parameter use_propagation: bool; default value: False

For propagation.

Parameter sigma_d_mask: float; default value: 0.05

Missing description for entry “sigma_d_mask”.

Parameter sigma_phi_mask: float; default value: 0.05

Missing description for entry “sigma_phi_mask”.

✎

Subscription velocity: topic ~velocity (Twist2DStamped)

Missing description for entry “velocity”.

Subscription segment_list: topic ~segment_list (SegmentList)

Missing description for entry “segment_list”.

✎

Publisher lane_pose: topic ~lane_pose (LanePose)

Missing description for entry “lane_pose”.

Publisher belief_img: topic ~belief_img (Image)

Missing description for entry “belief_img”.

Publisher entropy: topic ~entropy (Float32)

Missing description for entry “entropy”.

Publisher in_lane: topic ~in_lane (BoolStamped)

Missing description for entry “in_lane”.

Publisher switch: topic ~switch (BoolStamped)

Missing description for entry “switch”.

Because of mathjax bug
Because of mathjax bug

Package line_detector2

✎🔗

✎

Link to package on Github

✎

Andrea Censi (maintainer)

✎

A re-implementation of the line detector node.

This is a re-implementation of the package line_detector using the new facilities provided by easy_node.

line_detector_node2

✎🔗

(Generated from configuration line_detector_node2.easy_node.yaml.)

This is a rewriting of line_detector_node using the EasyNode framework.

✎

Parameter verbose: bool; default value: True

Whether the node is verbose or not. If set to True, the node will write timing statistics to the log.

Parameter img_size: not known

Missing description for entry “img_size”.

Parameter top_cutoff: int

This parameter decides how much of the image we should cut off. This is a performance improvement.

Parameter line_detector: str

This is the instance of line_detector to use.

✎

Subscription image: topic ~image (CompressedImage)

This is the compressed image to read. Note that it takes a long time to simply decode the image JPG.

The data is processed asynchronously in a different thread.

Subscription transform: topic ~transform (AntiInstagramTransform)

The anti-instagram transform to apply. See Unit Q-1 - Package anti_instagram.

Subscription switch: topic ~switch (BoolStamped)

This is a switch that allows to control the activity of this node. If the message is true, the node becomes active. If false, it switches off. The node starts as active.

✎

Publisher edge: topic ~edge (Image)

Missing description for entry “edge”.

Publisher color_segment: topic ~colorSegment (Image)

Missing description for entry “color_segment”.

Publisher segment_list: topic ~segment_list (SegmentList)

Missing description for entry “segment_list”.

Publisher image_with_lines: topic ~image_with_lines (Image)

Missing description for entry “image_with_lines”.

Because of mathjax bug
Because of mathjax bug

Package line_detector

✎🔗

✎

Link to package on Github

✎

Hang Zhao

Mack

✎

Add a description of package line_detector in package.xml.

This package is being replaced by the cleaned-up version, line_detector2. Do not write documentation here.

However, at the moment, all the launch files still call this one.

Because of mathjax bug

Packages - Indefinite navigation

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

AprilTags library

✎🔗

✎

Link to package on Github

✎

Michael Kaess

Hordur Johannson

Mitchell Wills

✎

A catkin version of the C++ apriltags library

Detect April tags (2D bar codes) in images; reports unique ID of each detection, and optionally its position and orientation relative to a calibrated camera.

See examples/apriltags_demo.cpp for a simple example that detects April tags (see tags/pdf/tag36h11.pdf) in laptop or webcam images and marks any tags in the live image.

Ubuntu dependencies: sudo apt-get install subversion cmake libopencv-dev libeigen3-dev libv4l-dev

Mac dependencies: sudo port install pkgconfig opencv eigen3

Uses the pods build system in connection with cmake, see: http://sourceforge.net/p/pods/

Michael Kaess October 2012

AprilTags were developed by Professor Edwin Olson of the University of Michigan. His Java implementation is available on this web site: http://april.eecs.umich.edu.

Olson’s Java code was ported to C++ and integrated into the Tekkotsu framework by Jeffrey Boyland and David Touretzky.

See this Tekkotsu wiki article for additional links and references: http://wiki.tekkotsu.org/index.php/AprilTags

This C++ code was further modified by Michael Kaess (kaess@mit.edu) and Hordur Johannson (hordurj@mit.edu) and the code has been released under the LGPL 2.1 license.

  • converted to standalone library
  • added stable homography recovery using OpenCV
  • robust tag code table that does not require a terminating 0 (omission results in false positives by illegal codes being accepted)
  • changed example tags to agree with Ed Olson’s Java version and added all his other tag families
  • added principal point as parameter as in original code - essential for homography
  • added some debugging code (visualization using OpenCV to show intermediate detection steps)
  • added fast approximation of arctan2 from Ed’s original Java code
  • using interpolation instead of homography in Quad: requires less homography computations and provides a small improvement in correct detections

todo: - significant speedup could be achieved by performing image operations using OpenCV (Gaussian filter, but also operations in TagDetector.cc) - replacing arctan2 by precomputed lookup table - converting matrix operations to Eigen (mostly for simplifying code, maybe some speedup)

Because of mathjax bug
Because of mathjax bug

Package apriltags_ros

✎🔗

AprilTags for ROS.

Build Status

Resource error
I will not embed remote files, such as https://api.travis-ci.org/RIVeR-Lab/apriltags_ros.png:

✎

Link to package on Github

✎

Mitchell Wills

Mitchell Wills

✎

A package that provides a ROS wrapper for the apriltags C++ package

Because of mathjax bug
Because of mathjax bug

Package fsm

✎🔗

✎

Link to package on Github

✎

Michael Misha Novitzky

Mack

✎

The finite state machine coordinates the modes of the car.

Node fsm_node

✎🔗

(Generated from configuration fsm_node.easy_node.yaml.)

Note that this node publishes on many topics according to the configuration:

for node_name, topic_name in nodes.items():
    self.pub_dict[node_name] = rospy.Publisher(topic_name, BoolStamped, ...)
✎

Parameter states: dict; default value: ``

Missing description for entry “states”.

Parameter nodes: not known

Missing description for entry “nodes”.

Parameter global_transitions: dict; default value: ``

Missing description for entry “global_transitions”.

Parameter initial_state: str; default value: ''

Missing description for entry “initial_state”.

Parameter events: dict; default value: ``

Missing description for entry “events”.

✎

No subscriptions defined.

✎

Publisher mode: topic ~mode (FSMState)

Missing description for entry “mode”.

Because of mathjax bug
Because of mathjax bug

Package indefinite_navigation

✎🔗

✎

Link to package on Github

✎

Liam Paull

✎

Add a description of package indefinite_navigation in package.xml.

Because of mathjax bug
Because of mathjax bug

Package intersection_control

✎🔗

✎

Link to package on Github

✎

Michael Misha Novitzky

Mack

✎

The intersection_control package implements a dead reckoning controller until we do something smarter.

Because of mathjax bug
Because of mathjax bug

Package stop_line_filter

✎🔗

✎

Link to package on Github

✎

Liam Paull

✎

Add a description of package stop_line_filter in package.xml.

Because of mathjax bug

Packages - Localization and planning

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package duckietown_description

✎🔗

✎

Link to package on Github

✎

Teddy Ort

Mack

✎

Add a description of package duckietown_description in package.xml.

Because of mathjax bug
Because of mathjax bug

Package localization

✎🔗

✎

Link to package on Github

✎

teddy

Mack

✎

Add a description of package localization in package.xml.

Because of mathjax bug

Packages - Coordination

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package led_detection

✎🔗

✎

Link to package on Github

✎

Andrea Censi

✎

Add a description of package led_detection in package.xml.

add authors

Package led_emitter

✎🔗

✎

Link to package on Github

✎

Andrea Censi

✎

description for led_emitter package.

Package led_interpreter

✎🔗

✎

Link to package on Github

✎

qlai

Mack

✎

Add a description of package led_interpreter in package.xml.

Because of mathjax bug
Because of mathjax bug

Package led_joy_mapper

✎🔗

✎

Link to package on Github

✎

lapentab

Mack

✎

Add a description of package led_joy_mapper in package.xml.

Because of mathjax bug
Because of mathjax bug

Package rgb_led

✎🔗

✎

Link to package on Github

✎

Dmitry Yershov

Mack

✎

Add a description of package rgb_led in package.xml.

Package traffic_light

✎🔗

✎

Link to package on Github

✎

Mack

✎

Add a description of package traffic_light in package.xml.

Because of mathjax bug

Packages - Additional functionality

✎🔗

to write

Because of mathjax bug
Because of mathjax bug

Package mdoap

✎🔗

✎

Link to package on Github

✎

Catherine Liu

Sam

Ari

Mack

✎

The mdoap package which handles object recognition and uses ground projection to estimate distances of objects for duckie safety. Also contains controllers for avoiding said obstacles

Because of mathjax bug
Because of mathjax bug

Package parallel_autonomy

✎🔗

✎

Link to package on Github

✎

Liam Paull

✎

description for the parallel autonomy package

Because of mathjax bug
Because of mathjax bug

Package vehicle_detection

✎🔗

✎

Link to package on Github

✎

Andrea Tacchetti

Mack

✎

Add a description of package vehicle_detection in package.xml.

Because of mathjax bug

Packages - Templates

✎🔗

These are templates.

Because of mathjax bug
Because of mathjax bug

Package pkg_name

✎🔗

✎

Link to package on Github

✎

Shih-Yuan Liu

Andrea Censi

✎

The package pkg_name is a template for ROS packages.

For the tutorial, see Unit K-6 - Minimal ROS node - pkg_name.

Package rostest_example

✎🔗

✎

Link to package on Github

✎

Teddy Ort

Mack

✎

Add a description of package rostest_example in package.xml.

Because of mathjax bug

Packages - Convenience

✎🔗

These packages are convenience packages that group together launch files and tests.

Because of mathjax bug
Because of mathjax bug

Package duckie_rr_bridge

✎🔗

✎

Link to package on Github

✎

greg

Mack

✎

The duckie_rr_bridge package. Creates a Robot Raconteur Service to drive the Duckiebot.

Because of mathjax bug
Because of mathjax bug

Package duckiebot_visualizer

✎🔗

✎

Link to package on Github

✎

Shih-Yuan Liu

Mack

✎

Add a description of package duckiebot_visualizer in package.xml.

Node duckiebot_visualizer

✎🔗

(Generated from configuration duckiebot_visualizer.easy_node.yaml.)

Missing node description in duckiebot_visualizer.easy_node.yaml.

✎

Parameter veh_name: str; default value: 'megaman'

Missing description for entry “veh_name”.

✎

Subscription seg_list: topic ~segment_list (SegmentList)

Missing description for entry “seg_list”.

✎

Publisher pub_seg_list: topic ~segment_list_markers (MarkerArray)

Missing description for entry “pub_seg_list”.

Because of mathjax bug
Because of mathjax bug

Package duckietown_demos

✎🔗

✎

Link to package on Github

✎

Liam Paull

✎

description of duckietown_demos

Because of mathjax bug
Because of mathjax bug

Package duckietown_logs

✎🔗

✎

Link to package on Github

✎

Andrea Censi

✎

Package to create videos from ROS logs.

Package duckietown_unit_test

✎🔗

✎

Link to package on Github

✎

Mack

✎

The duckietown_unit_test meta package contains all the unit test launch files for Duckietown.

Because of mathjax bug
Because of mathjax bug

Package veh_coordinator

✎🔗

✎

Link to package on Github

✎

Michal Cap

Mack

✎

The simple vehicle coordination package

I think is used to fake the vehicle coordination for the FSM.

Because of mathjax bug
Jeffrey Dean and Sanjay Ghemawat. Mapreduce: a flexible data processing tool. Commun. ACM, 53(1):72–77, 2010.    DOI  http  Gregory Dudek and Michael Jenkin. Computational principles of mobile robotics. Cambridge university press, 2010.    Rached Dhaouadi and A. Abu Hatab. Dynamic modeling of differential-drive mobile robots using lagrange and newton-euler methodologies: A unified framework. Advances in Robotics & Automation, 2(2):1–7, 2013.   

Last modified